-- Logs begin at Thu 2022-08-04 21:12:26 UTC, end at Thu 2022-08-04 22:47:16 UTC. -- Aug 04 22:06:30.096239 dv-cinder-852211-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 22:06:30.109073 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 22:06:30 2022] *** Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: nodename: dv-cinder-852211-1-iscsi Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: machine: x86_64 Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: clock source: unix Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: pcre jit disabled Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: detected number of CPU cores: 4 Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: current working directory: / Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: detected binary path: /usr/bin/uwsgi-core Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: your processes number limit is 47680 Aug 04 22:06:30.111793 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: your memory page size is 4096 bytes Aug 04 22:06:30.112385 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: detected max file descriptor number: 2048 Aug 04 22:06:30.112385 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: lock engine: pthread robust mutexes Aug 04 22:06:30.112385 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: thunder lock: enabled Aug 04 22:06:30.112385 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 22:06:30.112385 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 22:06:30.151477 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: Python main interpreter initialized at 0x55b72a8b99c0 Aug 04 22:06:30.151477 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: python threads support enabled Aug 04 22:06:30.151477 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: your server socket listen backlog is limited to 100 connections Aug 04 22:06:30.151477 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: your mercy for graceful operations on workers is 90 seconds Aug 04 22:06:30.151809 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 22:06:30.151882 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: *** Operational MODE: preforking *** Aug 04 22:06:30.152155 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: *** uWSGI is running in multiple interpreter mode *** Aug 04 22:06:30.152155 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: spawned uWSGI master process (pid: 91691) Aug 04 22:06:30.152349 dv-cinder-852211-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 22:06:30.153592 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: spawned uWSGI worker 1 (pid: 91701, cores: 1) Aug 04 22:06:30.153592 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: spawned uWSGI worker 2 (pid: 91702, cores: 1) Aug 04 22:06:30.153592 dv-cinder-852211-1-iscsi devstack@keystone.service[91691]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 22:06:35.649699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 22:06:35.649699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: from cryptography.utils import int_from_bytes Aug 04 22:06:35.650504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 22:06:35.650504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: from cryptography.utils import int_from_bytes Aug 04 22:06:35.650648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 22:06:35.650648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: from cryptography.utils import int_from_bytes Aug 04 22:06:35.651430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 22:06:35.651430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: from cryptography.utils import int_from_bytes Aug 04 22:06:35.700478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 22:06:35.700643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 22:06:35.700841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] command line args: [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 22:06:35.701107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 22:06:35.701258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 22:06:35.701385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ================================================================================ {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 22:06:35.701501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 22:06:35.701594 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] admin_token = **** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.701669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] command line args: [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 22:06:35.701787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] config_dir = [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.701956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 22:06:35.702048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.702144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ================================================================================ {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 22:06:35.702254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] config_source = [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.702335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] debug = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.702408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] admin_token = **** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.702483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.702821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] config_dir = [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.702821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.702939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] config_source = [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] debug = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] list_limit = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.703868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] log_config_append = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] log_dir = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] log_file = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] list_limit = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] log_config_append = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.704949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] log_dir = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] log_file = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.705974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.706124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.706276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.706394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.706540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.706647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.706786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.706894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.707006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.707176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.707308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.707419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.707525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.707661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.707780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.707893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.109/identity {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] publish_errors = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708738 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.708853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.109/identity {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] publish_errors = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709825 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] standard_threads = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.709977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] use_journal = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] standard_threads = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] use_json = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] use_stderr = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.710963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.711053 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] use_syslog = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.711131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.711207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.711287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.711358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] use_journal = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.711427 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.711503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] use_json = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.711609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.711682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] use_stderr = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.711812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.711897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] use_syslog = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.712046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 22:06:35.712131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.712290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.712373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.712525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.712609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.712769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.712855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.713025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.713104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.713286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.713796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.713880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.713967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.714971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.715081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.715210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.715318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.715461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.715587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.715696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.715806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.715960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.716040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.716174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.716273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.716413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.716524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.716668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.716783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.716922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.717034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.717175 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.717280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.717464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.717549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.717695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.717775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.717962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.718987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.719133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.719217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.719370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.719458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.719614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.719697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.719849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.719926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.720083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.720175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.720310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.720386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.720543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.720620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.720776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.720862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.721976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722055 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.722996 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.723149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.723226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.723382 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.723487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.723621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.723744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.723842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.723935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.724070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.724175 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.724304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.724432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.724548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.724661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.724770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.724883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.725026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.725156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.725271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.725385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.725683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.725801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.725914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.726963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.727085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.727239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.727358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.727471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.727592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.727706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.727819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.727944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.728048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.728160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.728282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.728396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.728509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.729117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.729654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.729807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.729935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.730981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.731094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.731214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.731338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.731453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.731569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.731682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.731796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.731914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732850 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.732964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.733916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.734200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.734317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.734430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.734543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.734662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.734777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.734901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.735969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.736088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.736202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.736315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.736427 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.736554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.736669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.736782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.736896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.737009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.737140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] auth.external = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.737289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.737429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.737561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.737561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.737784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.737784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] auth.password = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] auth.token = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] auth.external = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.738990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.739105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.739218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.739333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.739447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.739568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.739684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] auth.password = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.739797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.739911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] auth.token = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.740025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.740139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.740260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] credential.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.740376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.740490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.740663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.740807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.740924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.741048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.741164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.741278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.741392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.741540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.741658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.741786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.741903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] credential.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742138 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.742946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.743065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.743190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.743305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.743419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.743541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.743656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.743776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.743903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.744019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.744134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] federation.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.744248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.744994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.745376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.745586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.745734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.745878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.745878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.746145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.746286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.746439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.746579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.746724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.746724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] federation.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.746986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.747134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.747281 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.747452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.747591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.747729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.747867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.748004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.748154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.748297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.748437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.748581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.748718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.748855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.749001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.749139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.749276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.749276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.749610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.749828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.749969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.750107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.750245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.750391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.750530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.750690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.750846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.750991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.751129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.751266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.751404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.751551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.751689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.751923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.752075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.752214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.752359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.752507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.752647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.752785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.752923 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.753061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.753199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.753347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.753532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.753679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.753824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.753967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.754111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.754268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.754419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.754563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.754707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.754851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.754997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.755150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.755295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.755443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.755587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.755733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.755891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.756044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.756189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.756346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.756491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.756634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.756779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.756935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.757080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.757224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.757367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.757554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.757698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.757853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.757998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.758140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.758282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.758428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.758570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.758720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.758865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.759006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.759148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.759291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.759432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.759581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.759725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.759905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.759905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.760184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.760326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.760512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.760691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.760834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.760978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.761124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.761274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.761504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.761648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.761789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.761927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.762067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.762207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.762358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.762502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.762641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.762795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.762934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.763072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.763220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.763359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.763503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.763642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.763779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.763918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.764066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.764205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.764343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.764487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.764626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.764773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.764911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.765048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.765187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.765326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.765495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.765653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.765806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.765950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.765950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.766221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.766369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.766523 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.766666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.766807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.766948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.767090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.767234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.767384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.767562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.767709 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.767851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.767992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.768134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.768286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.768433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.768575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.768718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.768859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.769000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.769151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.769294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.769464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.769607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.769747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.769896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.770046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.770185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.770325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.770471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.770868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.771011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.771159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.771305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.771445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.771585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.771724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.771864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.772011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.772150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.772290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.772446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.772596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.772740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.772892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.773035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.773176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.773317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.773317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.773625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.773776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.773915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.774054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] resource.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.774191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.774330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.774468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.774620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.774760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.774899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.775037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.775175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.775317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.775466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.775641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.775780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.775918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.776056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.776193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.776340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.776479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.776622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.776762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.776901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] role.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.777040 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] resource.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.777187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] role.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.777325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.777492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.777675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.777822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.777967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.778187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.778335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.778481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.778625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.778776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.778921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.779074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.779220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.779364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.779508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.779652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.779804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.779950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.780094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] role.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.780237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.780381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] role.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.780525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.780682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.780827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.780972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.781116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.781259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.781470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.781630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.781630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.781898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.782036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.782175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.782312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.782461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.782606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.782745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.782883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.783022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.783160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.783306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.783306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.783569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.783707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.783844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.783982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.784128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.784268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.784411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.784622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.784761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.784899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.785046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.785185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.785328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.785491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.785637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.785637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.785919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.786064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.786208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.786349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.786498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.786650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.786793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.786934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.787076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.787220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.787361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] token.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.787511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.787654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.787827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.787970 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.788112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.788256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.788416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.788560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.788702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.788843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.788984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.789125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.789276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.789483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.789628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.789768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.789919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.790065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.790216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] token.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.790357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.790357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.790624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.790763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.790900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.791047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.791756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.791912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.792050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.792188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.792486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.792638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.792778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.792916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.793061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.793200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.connection = **** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.793348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.793525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.793679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.793835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.793979 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.794121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.794263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.794405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.794547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.794699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.794842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.794985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.795134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.795277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.795419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.795572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.795715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.795857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.796003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.796145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.796288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.connection = **** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.796442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.796442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.796849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.796992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.797134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.797282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.797462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.797618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.797767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.797939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.798959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.800140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.800140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 22:06:35.800140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 22:06:35.815938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.816222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.816370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.817584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.817830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.818064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.824661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.826881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.854353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.854577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.855417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.855657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.907215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.907903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.958151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.958395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.959017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:35.959259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.022936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.023140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.023273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.023399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.023521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.023660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.023783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.023904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.024022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.024141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.024258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.024258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.047549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.047721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.047889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.048037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.048176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.048308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.048428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.048546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.048665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.048790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.048911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.049028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.049144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.049261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.060433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.060725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.073177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.073442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.073670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.073795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.073931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.074992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 22:06:36.092420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.092603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.092735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 22:06:36.092859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 22:06:36.092980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.093097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.093230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 22:06:36.093352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 22:06:36.093486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.093604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.093720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.093849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.093969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.094085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.094201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.094317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.105156 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.105818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.105948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.106096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.106207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.106331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.106447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.106551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.112188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.112338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.116950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.117110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.117238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.117390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.117541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.117698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.117835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.117981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.118103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.118262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.118382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.118509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.118627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.118776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.118897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.119040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.119163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.119366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.119664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.119664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.120189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.120306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.142462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.143321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.143927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.144202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.144817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.145100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.149045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.149315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.149615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.150030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.150311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.151479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.156708 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.156978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.157983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.158333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.161072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.161363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.161666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.161953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.162236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.162482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.162608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.162762 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.162882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.163024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.163154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.163312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.163603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.163895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.164167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.164436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.179402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.179675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.181154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.181456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.183854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.185559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.187000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.187291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.188705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.189003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.191660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.193509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.195171 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.197101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.198433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.198725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.199013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.199292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.199565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.199845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.200357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.200658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.200942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.201221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.201540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.201817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.216723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.216997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.217255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.217544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.220498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.220789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.221060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.221327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.224356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.225928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.226199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.228234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.229648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.229921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.230205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.230483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.230622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.230749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.230870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.233674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.233964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.234270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.234557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.234838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.242263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.242529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.242807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.243084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.243359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.243627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.243904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.246421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.246704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.246987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.247267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.247545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.247825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.248107 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.259555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.262910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.264073 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.265847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.267082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.268413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.269921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.270047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.270209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.270485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.272816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.274392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.274680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.274962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.277825 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.279420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.282171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.282358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.282641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.282919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.283191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.283815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.286766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.287061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.287349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.287629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.292252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.292252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.292541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.296365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.296656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.296942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.299292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.299567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.299843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.300114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.300380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.300644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.300908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.301185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.301603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.301876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.302147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.302408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.302672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.302939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.303868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 22:06:36.304153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.304439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.304719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.304991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.305261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.305558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.305854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.306247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.306522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.306916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.306916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.307493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.307671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 22:06:36.336199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b72a8b99c0 pid: 91701 (default app) Aug 04 22:06:36.340127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106b4ff-38e9-4edb-ae74-2f5089349662 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:36.340408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106b4ff-38e9-4edb-ae74-2f5089349662 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:36.340682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106b4ff-38e9-4edb-ae74-2f5089349662 None None] PATH_INFO: `/v3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:36.341821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b72a8b99c0 pid: 91702 (default app) Aug 04 22:06:36.344953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1/1] 10.222.0.109 () {50 vars in 796 bytes} [Thu Aug 4 22:06:36 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:43.985731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dde8f-f668-4cbb-8651-72631d0b452d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:43.986306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dde8f-f668-4cbb-8651-72631d0b452d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:43.986306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dde8f-f668-4cbb-8651-72631d0b452d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:43.990716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1/2] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:43 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:06:43.996782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:43.997228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:43.997322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:44.068403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: INFO dbcounter [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] Registered counter for database keystone Aug 04 22:06:44.087576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG oslo_db.sqlalchemy.engines [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91701) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 22:06:44.094023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writer thread running {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 22:06:44.273633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG passlib.handlers.bcrypt [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91701) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 22:06:44.273973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG passlib.handlers.bcrypt [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91701) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 22:06:44.330903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:44.332526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:44.462552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 2/3] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:43 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 22:06:44.630907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dde8f-f668-4cbb-8651-72631d0b452d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:44.703212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: INFO dbcounter [None req-3e1dde8f-f668-4cbb-8651-72631d0b452d None None] Registered counter for database keystone Aug 04 22:06:44.725021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG oslo_db.sqlalchemy.engines [None req-3e1dde8f-f668-4cbb-8651-72631d0b452d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91702) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 22:06:44.731635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writer thread running {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 22:06:44.853108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dde8f-f668-4cbb-8651-72631d0b452d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:44.857909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318f8b8-ad47-47ff-a364-852a610accbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:44.858546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318f8b8-ad47-47ff-a364-852a610accbb admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:44.858823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318f8b8-ad47-47ff-a364-852a610accbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:44.859093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318f8b8-ad47-47ff-a364-852a610accbb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:44.867130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318f8b8-ad47-47ff-a364-852a610accbb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:44.921383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9318f8b8-ad47-47ff-a364-852a610accbb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 22:06:44.921690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318f8b8-ad47-47ff-a364-852a610accbb admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:44.931187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 2/4] 10.222.0.109 () {58 vars in 1329 bytes} [Thu Aug 4 22:06:44 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:06:44.937160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:44.952057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e422992-e9da-4713-96e0-b3da3e29baf7 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:44.956494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498826a9-b306-4538-a5bd-5ee48db71186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:44.957650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498826a9-b306-4538-a5bd-5ee48db71186 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:44.957937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498826a9-b306-4538-a5bd-5ee48db71186 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:44.958213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498826a9-b306-4538-a5bd-5ee48db71186 admin admin] PATH_INFO: `/v3/projects/ee8a7cb4ed304a5598e10ca71a963013` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:44.959693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498826a9-b306-4538-a5bd-5ee48db71186 admin admin] RBAC: Authorizing `identity:get_project(project_id=ee8a7cb4ed304a5598e10ca71a963013)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:45.011298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING keystone.common.rbac_enforcer.enforcer [None req-498826a9-b306-4538-a5bd-5ee48db71186 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 22:06:45.011847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498826a9-b306-4538-a5bd-5ee48db71186 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:45.016304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 3/5] 10.222.0.109 () {56 vars in 1212 bytes} [Thu Aug 4 22:06:44 2022] GET /identity/v3/projects/ee8a7cb4ed304a5598e10ca71a963013 => generated 345 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:47.007087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de167d-815f-4824-93f0-74762f7f8a53 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:47.007638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de167d-815f-4824-93f0-74762f7f8a53 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:47.007638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de167d-815f-4824-93f0-74762f7f8a53 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:47.012489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 3/6] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:47 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:47.018132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4d9e3-67bf-45be-9e95-0d1c46ff1150 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:47.018544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4d9e3-67bf-45be-9e95-0d1c46ff1150 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:47.018686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4d9e3-67bf-45be-9e95-0d1c46ff1150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:47.039350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-cfa4d9e3-67bf-45be-9e95-0d1c46ff1150 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:47.041054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-cfa4d9e3-67bf-45be-9e95-0d1c46ff1150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:47.048868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 4/7] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:47 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 22:06:47.070439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de167d-815f-4824-93f0-74762f7f8a53 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:47.082983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de167d-815f-4824-93f0-74762f7f8a53 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:47.087191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c0ccf-13af-4117-908f-e1e816982555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:47.088478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c0ccf-13af-4117-908f-e1e816982555 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:47.088767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c0ccf-13af-4117-908f-e1e816982555 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:47.089042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c0ccf-13af-4117-908f-e1e816982555 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:47.096737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c0ccf-13af-4117-908f-e1e816982555 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:47.098561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c0ccf-13af-4117-908f-e1e816982555 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:47.107566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 4/8] 10.222.0.109 () {58 vars in 1329 bytes} [Thu Aug 4 22:06:47 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:06:47.113221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4d9e3-67bf-45be-9e95-0d1c46ff1150 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:47.122565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4d9e3-67bf-45be-9e95-0d1c46ff1150 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:47.126990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780efc32-8adb-48ea-9e24-a7f20ffbd184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:47.127630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780efc32-8adb-48ea-9e24-a7f20ffbd184 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:47.127913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780efc32-8adb-48ea-9e24-a7f20ffbd184 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:47.128190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780efc32-8adb-48ea-9e24-a7f20ffbd184 admin admin] PATH_INFO: `/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:47.129587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780efc32-8adb-48ea-9e24-a7f20ffbd184 admin admin] RBAC: Authorizing `identity:get_user(user_id=f59916fe4d6d4a6aac1fc1ff790ac163)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:47.131434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780efc32-8adb-48ea-9e24-a7f20ffbd184 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:47.132992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 5/9] 10.222.0.109 () {56 vars in 1203 bytes} [Thu Aug 4 22:06:47 2022] GET /identity/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163 => generated 248 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:49.183009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e298d3b-564d-4e33-9487-c789292a611e None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.183535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e298d3b-564d-4e33-9487-c789292a611e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.183535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e298d3b-564d-4e33-9487-c789292a611e None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.184891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 5/10] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:49.190063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534995ab-8b15-4a24-9bbb-8a1eb47ee77a None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.190457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534995ab-8b15-4a24-9bbb-8a1eb47ee77a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.190734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534995ab-8b15-4a24-9bbb-8a1eb47ee77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.210182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-534995ab-8b15-4a24-9bbb-8a1eb47ee77a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:49.211465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-534995ab-8b15-4a24-9bbb-8a1eb47ee77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:49.230541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef52bb4-6090-45f4-af6d-35e032ff4d1c None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.230829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef52bb4-6090-45f4-af6d-35e032ff4d1c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.231107 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef52bb4-6090-45f4-af6d-35e032ff4d1c None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.232422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 6/11] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:06:49.237327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.237644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.237920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.279468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG passlib.handlers.bcrypt [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91702) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 22:06:49.279894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG passlib.handlers.bcrypt [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91702) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 22:06:49.291751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 6/12] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:06:49.314884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534995ab-8b15-4a24-9bbb-8a1eb47ee77a None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:49.326071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:49.327676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:49.338153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:49.419443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534995ab-8b15-4a24-9bbb-8a1eb47ee77a None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:49.430730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 7/13] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:06:49.453387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:49.453617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86630b25-7a40-497e-8421-2690c323a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:49.454120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630b25-7a40-497e-8421-2690c323a79f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.454406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630b25-7a40-497e-8421-2690c323a79f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.454687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86630b25-7a40-497e-8421-2690c323a79f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.465951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86630b25-7a40-497e-8421-2690c323a79f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-86630b25-7a40-497e-8421-2690c323a79f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:06:49.474763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:06:49.475880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:06:49.476731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 7/14] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:06:49.482142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86630b25-7a40-497e-8421-2690c323a79f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:49.554656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c4fb4-bada-4b3b-be04-deb9543df051 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:49.557550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:49.563102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:49.582951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86630b25-7a40-497e-8421-2690c323a79f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:49.594515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92541f-2d72-4f49-8643-9095a6c1c3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:49.595377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92541f-2d72-4f49-8643-9095a6c1c3b0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.595658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92541f-2d72-4f49-8643-9095a6c1c3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.595925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92541f-2d72-4f49-8643-9095a6c1c3b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.617002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:49.617651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.617932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.618205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.619470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:06:49.619774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:49.621187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:49.632678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 8/15] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:06:49 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:49.638857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:49.664704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92541f-2d72-4f49-8643-9095a6c1c3b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:49.666469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92541f-2d72-4f49-8643-9095a6c1c3b0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:49.768724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 8/16] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:06:49 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 22:06:49.774810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92541f-2d72-4f49-8643-9095a6c1c3b0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:49.778277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:49.845220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f484f5d-43d4-45a8-9a1b-2f3c77d51d89 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:49.877923 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92541f-2d72-4f49-8643-9095a6c1c3b0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:49.878577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d619944-21fa-4522-96e2-602862ff641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:49.879194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d619944-21fa-4522-96e2-602862ff641c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.879475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d619944-21fa-4522-96e2-602862ff641c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.879742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d619944-21fa-4522-96e2-602862ff641c None admin] PATH_INFO: `/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.880905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d619944-21fa-4522-96e2-602862ff641c None admin] RBAC: Authorizing `identity:get_user(user_id=f59916fe4d6d4a6aac1fc1ff790ac163)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:49.882391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d619944-21fa-4522-96e2-602862ff641c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:49.883971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 9/17] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:06:49 2022] GET /identity/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163 => generated 248 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:49.889937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d619944-21fa-4522-96e2-602862ff641c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:49.904428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:49.913824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:49.914698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:49.914972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:49.915239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:49.934135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:49.935591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:06:49.953856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:06:49.954716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:06:49.955504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:06:49.955504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:06:49.955504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:06:49.955504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:06:49.955504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 9/18] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:06:49.960972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:49.991158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d619944-21fa-4522-96e2-602862ff641c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:50.018312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:50.026593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec525130-4026-4902-803a-5b55e622c7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:50.027187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec525130-4026-4902-803a-5b55e622c7a4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:50.027460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec525130-4026-4902-803a-5b55e622c7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:50.027733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec525130-4026-4902-803a-5b55e622c7a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:50.028896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec525130-4026-4902-803a-5b55e622c7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:50.030404 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec525130-4026-4902-803a-5b55e622c7a4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:50.032026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 10/19] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:06:49 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:50.037838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec525130-4026-4902-803a-5b55e622c7a4 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:50.065238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3501dc6-8900-40eb-ae30-333ec8ae05e6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:50.097961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954163e-bcd9-4ac4-b81c-e138bf139074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:50.098571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954163e-bcd9-4ac4-b81c-e138bf139074 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:50.098847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954163e-bcd9-4ac4-b81c-e138bf139074 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:50.099149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954163e-bcd9-4ac4-b81c-e138bf139074 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:50.099839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954163e-bcd9-4ac4-b81c-e138bf139074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:06:50.100129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954163e-bcd9-4ac4-b81c-e138bf139074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:50.101537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954163e-bcd9-4ac4-b81c-e138bf139074 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:50.120999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 10/20] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:06:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:50.125112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:50.146989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec525130-4026-4902-803a-5b55e622c7a4 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:50.184113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e7fb9-09d9-41ff-9f06-e7f6c56c2361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:50.184755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e7fb9-09d9-41ff-9f06-e7f6c56c2361 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:50.185039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e7fb9-09d9-41ff-9f06-e7f6c56c2361 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:50.185317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e7fb9-09d9-41ff-9f06-e7f6c56c2361 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:50.186244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e7fb9-09d9-41ff-9f06-e7f6c56c2361 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.domain.id=default, user.id=f59916fe4d6d4a6aac1fc1ff790ac163) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:06:50.186556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e7fb9-09d9-41ff-9f06-e7f6c56c2361 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:50.188070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e7fb9-09d9-41ff-9f06-e7f6c56c2361 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:50.195636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 11/21] 10.222.0.109 () {56 vars in 1327 bytes} [Thu Aug 4 22:06:50 2022] GET /identity/v3/role_assignments?user.id=f59916fe4d6d4a6aac1fc1ff790ac163&scope.domain.id=default&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 234 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:52.527946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4217b-f2dc-4e19-9751-95a5ec29804d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:52.528445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4217b-f2dc-4e19-9751-95a5ec29804d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:52.528445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4217b-f2dc-4e19-9751-95a5ec29804d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:52.530104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 11/22] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:52.535323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81b544-e2a9-47e3-b615-7b61844e1a53 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:52.535716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81b544-e2a9-47e3-b615-7b61844e1a53 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:52.535879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81b544-e2a9-47e3-b615-7b61844e1a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:52.558061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7a81b544-e2a9-47e3-b615-7b61844e1a53 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:52.559276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7a81b544-e2a9-47e3-b615-7b61844e1a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:52.651370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 12/23] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:06:52.686648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81b544-e2a9-47e3-b615-7b61844e1a53 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:52.686846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099344a1-fdb9-4c48-8e60-12b44aebeea5 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:52.687138 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099344a1-fdb9-4c48-8e60-12b44aebeea5 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:52.687420 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099344a1-fdb9-4c48-8e60-12b44aebeea5 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:52.688932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 12/24] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:52.726042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f361a-fa14-434c-aef8-981622a7cf8d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:52.726334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f361a-fa14-434c-aef8-981622a7cf8d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:52.726606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f361a-fa14-434c-aef8-981622a7cf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:52.762457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-929f361a-fa14-434c-aef8-981622a7cf8d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:52.763872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-929f361a-fa14-434c-aef8-981622a7cf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:52.777848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:52.820695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81b544-e2a9-47e3-b615-7b61844e1a53 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:52.874786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 13/25] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:06:52.888889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91451d86-f1c1-4ec4-b387-f0ff994b0919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:52.889582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451d86-f1c1-4ec4-b387-f0ff994b0919 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:52.889870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451d86-f1c1-4ec4-b387-f0ff994b0919 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:52.890152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451d86-f1c1-4ec4-b387-f0ff994b0919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:52.906087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91451d86-f1c1-4ec4-b387-f0ff994b0919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:52.907613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91451d86-f1c1-4ec4-b387-f0ff994b0919 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:52.914250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f361a-fa14-434c-aef8-981622a7cf8d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-91451d86-f1c1-4ec4-b387-f0ff994b0919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:06:52.935642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:06:52.936476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:06:52.937261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:06:52.937261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:06:52.937261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:06:52.937261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:06:52.939181 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 13/26] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:06:52.940674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40d9f5-3a90-4d1b-b10b-345fa03385b6 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:52.941174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40d9f5-3a90-4d1b-b10b-345fa03385b6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:52.941686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40d9f5-3a90-4d1b-b10b-345fa03385b6 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:52.943483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 14/27] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:52.944810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef531e-18d6-4068-90e6-aa3bcc32c6fa None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:52.945307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef531e-18d6-4068-90e6-aa3bcc32c6fa None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:52.945811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef531e-18d6-4068-90e6-aa3bcc32c6fa None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:52.948103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 15/28] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:52.952120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef531e-18d6-4068-90e6-aa3bcc32c6fa None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:52.971344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:53.037530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f361a-fa14-434c-aef8-981622a7cf8d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:53.082087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506b14e-dd57-41ae-bfe4-d9aebe8d2579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:53.082834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506b14e-dd57-41ae-bfe4-d9aebe8d2579 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:53.083130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506b14e-dd57-41ae-bfe4-d9aebe8d2579 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:53.083515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506b14e-dd57-41ae-bfe4-d9aebe8d2579 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:53.084258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506b14e-dd57-41ae-bfe4-d9aebe8d2579 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:53.084887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef531e-18d6-4068-90e6-aa3bcc32c6fa None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:53.085707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506b14e-dd57-41ae-bfe4-d9aebe8d2579 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:53.127424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:53.139200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33dd06-1139-4591-8704-ae5fafcbadff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:53.140163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33dd06-1139-4591-8704-ae5fafcbadff None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:53.140590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33dd06-1139-4591-8704-ae5fafcbadff None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:53.141011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33dd06-1139-4591-8704-ae5fafcbadff None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:53.141946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33dd06-1139-4591-8704-ae5fafcbadff None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:06:53.142394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33dd06-1139-4591-8704-ae5fafcbadff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:53.144114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33dd06-1139-4591-8704-ae5fafcbadff None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:53.166752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 16/29] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:06:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:53.168413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141e27-6c3c-41ed-acf5-134f8cb919bf None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:53.168848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141e27-6c3c-41ed-acf5-134f8cb919bf None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:53.169267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141e27-6c3c-41ed-acf5-134f8cb919bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:53.193912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-06141e27-6c3c-41ed-acf5-134f8cb919bf None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:53.195154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-06141e27-6c3c-41ed-acf5-134f8cb919bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:53.285544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 17/30] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:06:53.286849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29188-4c22-4443-91c4-8898272c04c8 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:53.287139 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29188-4c22-4443-91c4-8898272c04c8 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:53.287420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29188-4c22-4443-91c4-8898272c04c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:53.319327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-baa29188-4c22-4443-91c4-8898272c04c8 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:53.320510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-baa29188-4c22-4443-91c4-8898272c04c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:53.353578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 14/31] 10.222.0.109 () {60 vars in 1136 bytes} [Thu Aug 4 22:06:52 2022] POST /identity/v3/roles => generated 220 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:06:53.354740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506b14e-dd57-41ae-bfe4-d9aebe8d2579 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:53.444376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 18/32] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:06:53.445533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa29188-4c22-4443-91c4-8898272c04c8 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:53.545760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:53.560640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506b14e-dd57-41ae-bfe4-d9aebe8d2579 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:53.578147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:53.611924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefefd1b-0e24-4609-bb9a-cdf1154fe105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:53.612902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefefd1b-0e24-4609-bb9a-cdf1154fe105 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:53.613516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefefd1b-0e24-4609-bb9a-cdf1154fe105 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:53.614009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefefd1b-0e24-4609-bb9a-cdf1154fe105 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:53.615219 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefefd1b-0e24-4609-bb9a-cdf1154fe105 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:53.617813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefefd1b-0e24-4609-bb9a-cdf1154fe105 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:53.627531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa29188-4c22-4443-91c4-8898272c04c8 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:53.658629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 15/33] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:06:53 2022] POST /identity/v3/projects => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:06:53.659801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefefd1b-0e24-4609-bb9a-cdf1154fe105 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:53.674772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9986171-688a-4440-9ba5-19d9049ae8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:53.675860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9986171-688a-4440-9ba5-19d9049ae8e6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:53.676357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9986171-688a-4440-9ba5-19d9049ae8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:53.676890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9986171-688a-4440-9ba5-19d9049ae8e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:53.691323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9986171-688a-4440-9ba5-19d9049ae8e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-f9986171-688a-4440-9ba5-19d9049ae8e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:06:53.699339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:06:53.700275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:06:53.701063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:06:53.703832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 19/34] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:06:53.710573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9986171-688a-4440-9ba5-19d9049ae8e6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:53.782477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefefd1b-0e24-4609-bb9a-cdf1154fe105 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:53.819999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17099e3c-eead-4be1-852f-ee43a761768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:53.820635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099e3c-eead-4be1-852f-ee43a761768e None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:53.820968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099e3c-eead-4be1-852f-ee43a761768e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:53.821247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099e3c-eead-4be1-852f-ee43a761768e None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:53.822675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17099e3c-eead-4be1-852f-ee43a761768e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:53.824234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17099e3c-eead-4be1-852f-ee43a761768e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:53.834288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9986171-688a-4440-9ba5-19d9049ae8e6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:53.846413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:53.850870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 16/35] 10.222.0.109 () {60 vars in 1136 bytes} [Thu Aug 4 22:06:53 2022] POST /identity/v3/roles => generated 226 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:06:53.878307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:53.878872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:53.879214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:53.879498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:53.880260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:06:53.880587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:53.882216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:53.902638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 20/36] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:06:53 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:53.908544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17099e3c-eead-4be1-852f-ee43a761768e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:53.936048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:54.040020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17099e3c-eead-4be1-852f-ee43a761768e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:54.082338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c58b6-6428-4b1f-8f20-cb5d53c1d5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:54.083355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c58b6-6428-4b1f-8f20-cb5d53c1d5ea None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.083852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c58b6-6428-4b1f-8f20-cb5d53c1d5ea None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.084324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c58b6-6428-4b1f-8f20-cb5d53c1d5ea None admin] PATH_INFO: `/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.085959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c58b6-6428-4b1f-8f20-cb5d53c1d5ea None admin] RBAC: Authorizing `identity:get_user(user_id=f59916fe4d6d4a6aac1fc1ff790ac163)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:54.088180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c58b6-6428-4b1f-8f20-cb5d53c1d5ea None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:54.090370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 17/37] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:06:53 2022] GET /identity/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163 => generated 248 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:54.096173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:54.107795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de1856-0d8c-4e41-a028-ab64f68f180c None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.108131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de1856-0d8c-4e41-a028-ab64f68f180c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.108352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de1856-0d8c-4e41-a028-ab64f68f180c None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.109946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 18/38] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:06:54.116041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2eb4e-8f1d-43e0-bdab-0890e3a1709c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.116336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2eb4e-8f1d-43e0-bdab-0890e3a1709c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.116614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2eb4e-8f1d-43e0-bdab-0890e3a1709c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.139282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f1c2eb4e-8f1d-43e0-bdab-0890e3a1709c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:54.140737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f1c2eb4e-8f1d-43e0-bdab-0890e3a1709c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:54.214338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 19/39] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:06:54.215642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6defc6-284a-47e1-9ef5-80f426487cfe None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.215953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6defc6-284a-47e1-9ef5-80f426487cfe None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.216256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6defc6-284a-47e1-9ef5-80f426487cfe None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.217674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 20/40] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:54.222863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c739-f6bb-4525-97d7-90ed254f83b5 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.223314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c739-f6bb-4525-97d7-90ed254f83b5 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.223444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c739-f6bb-4525-97d7-90ed254f83b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.226027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4f88f-2b36-4329-82f6-0987ede2168f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:54.242933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8264c739-f6bb-4525-97d7-90ed254f83b5 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:54.244586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8264c739-f6bb-4525-97d7-90ed254f83b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:54.255888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:54.261154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19540ed-3d68-4f39-aeae-80b5e14d65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:54.261827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19540ed-3d68-4f39-aeae-80b5e14d65f5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.262108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19540ed-3d68-4f39-aeae-80b5e14d65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.262376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19540ed-3d68-4f39-aeae-80b5e14d65f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.263520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19540ed-3d68-4f39-aeae-80b5e14d65f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:54.264973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19540ed-3d68-4f39-aeae-80b5e14d65f5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:54.266587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 21/41] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:06:54 2022] GET /identity/v3/domains/default => generated 200 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:54.267316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19540ed-3d68-4f39-aeae-80b5e14d65f5 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:54.302440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:54.316178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 21/42] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:06:54.317005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264c739-f6bb-4525-97d7-90ed254f83b5 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:54.371249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19540ed-3d68-4f39-aeae-80b5e14d65f5 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:54.406264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3ff30-d82d-4dde-9625-79588820c5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:54.406811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3ff30-d82d-4dde-9625-79588820c5a7 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.407088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3ff30-d82d-4dde-9625-79588820c5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.407358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3ff30-d82d-4dde-9625-79588820c5a7 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.408065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3ff30-d82d-4dde-9625-79588820c5a7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:54.409465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3ff30-d82d-4dde-9625-79588820c5a7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:54.419291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264c739-f6bb-4525-97d7-90ed254f83b5 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:54.454201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523daa5c-c362-46f5-9de7-8402898b86c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:54.454875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523daa5c-c362-46f5-9de7-8402898b86c2 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.455156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523daa5c-c362-46f5-9de7-8402898b86c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.455431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523daa5c-c362-46f5-9de7-8402898b86c2 None admin] PATH_INFO: `/v3/domains/default/users/f59916fe4d6d4a6aac1fc1ff790ac163/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.457438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523daa5c-c362-46f5-9de7-8402898b86c2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f59916fe4d6d4a6aac1fc1ff790ac163, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:54.458855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523daa5c-c362-46f5-9de7-8402898b86c2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:54.474140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 22/43] 10.222.0.109 () {58 vars in 1366 bytes} [Thu Aug 4 22:06:54 2022] PUT /identity/v3/domains/default/users/f59916fe4d6d4a6aac1fc1ff790ac163/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:06:54.475140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523daa5c-c362-46f5-9de7-8402898b86c2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:54.485726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:54.532663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 22/44] 10.222.0.109 () {60 vars in 1136 bytes} [Thu Aug 4 22:06:54 2022] POST /identity/v3/roles => generated 224 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:06:54.597870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523daa5c-c362-46f5-9de7-8402898b86c2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:54.643352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee46cbb-a02b-4621-a17a-091c211e7abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:54.644030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee46cbb-a02b-4621-a17a-091c211e7abb None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.644310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee46cbb-a02b-4621-a17a-091c211e7abb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.644584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee46cbb-a02b-4621-a17a-091c211e7abb None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.646082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee46cbb-a02b-4621-a17a-091c211e7abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:54.647560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee46cbb-a02b-4621-a17a-091c211e7abb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:54.649525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 23/45] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:06:54 2022] GET /identity/v3/domains/default => generated 200 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:54.656570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3ff30-d82d-4dde-9625-79588820c5a7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:54.762261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:54.768475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3ff30-d82d-4dde-9625-79588820c5a7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:54.804204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f844d2a-67be-4c53-b0d3-88ab9d75d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:54.804832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f844d2a-67be-4c53-b0d3-88ab9d75d466 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:54.805115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f844d2a-67be-4c53-b0d3-88ab9d75d466 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:54.805387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f844d2a-67be-4c53-b0d3-88ab9d75d466 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:54.806131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f844d2a-67be-4c53-b0d3-88ab9d75d466 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:54.807712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f844d2a-67be-4c53-b0d3-88ab9d75d466 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:54.835458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 23/46] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:06:54 2022] POST /identity/v3/projects => generated 313 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:06:56.522977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6923-b39f-48e7-9111-c36a54428bad None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:56.523467 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6923-b39f-48e7-9111-c36a54428bad None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:56.523467 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6923-b39f-48e7-9111-c36a54428bad None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:56.525345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 24/47] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:56 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:56.530551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da6b08-cd91-4f7c-9be1-8a786c90c75a None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:56.530990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da6b08-cd91-4f7c-9be1-8a786c90c75a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:56.531141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da6b08-cd91-4f7c-9be1-8a786c90c75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:56.550870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-87da6b08-cd91-4f7c-9be1-8a786c90c75a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:56.552408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-87da6b08-cd91-4f7c-9be1-8a786c90c75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:56.617531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 24/48] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:06:56.640574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6923-b39f-48e7-9111-c36a54428bad None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:56.716104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:56.742406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6923-b39f-48e7-9111-c36a54428bad None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:56.776574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df05f4-a01a-4e3b-b7f2-8097b2f05b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:56.777129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df05f4-a01a-4e3b-b7f2-8097b2f05b7c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:56.777419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df05f4-a01a-4e3b-b7f2-8097b2f05b7c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:56.777704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df05f4-a01a-4e3b-b7f2-8097b2f05b7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:56.788900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df05f4-a01a-4e3b-b7f2-8097b2f05b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-28df05f4-a01a-4e3b-b7f2-8097b2f05b7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:06:56.795198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:06:56.796038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:06:56.796864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 25/49] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:06:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:06:56.802299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da6b08-cd91-4f7c-9be1-8a786c90c75a None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:56.864150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769ad12-bf37-4e9e-909f-bd7d5f3cdd16 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:56.864431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769ad12-bf37-4e9e-909f-bd7d5f3cdd16 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:56.864712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769ad12-bf37-4e9e-909f-bd7d5f3cdd16 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:56.866092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 26/50] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:56.871303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158b427-ff7d-47ba-9daf-37826a22ff4a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:56.871591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158b427-ff7d-47ba-9daf-37826a22ff4a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:56.871862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158b427-ff7d-47ba-9daf-37826a22ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:56.879653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:56.890947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5158b427-ff7d-47ba-9daf-37826a22ff4a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:56.892154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5158b427-ff7d-47ba-9daf-37826a22ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:56.903911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da6b08-cd91-4f7c-9be1-8a786c90c75a None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:56.937395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:56.938039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:56.938319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:56.938593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:56.939294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:06:56.939596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:56.940981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:56.949943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 25/51] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:06:56 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:56.955211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:56.959308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 27/52] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:06:56.982093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158b427-ff7d-47ba-9daf-37826a22ff4a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:57.044635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:57.055254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62128141-fd22-4843-aedf-96354e1443cd None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:57.082981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158b427-ff7d-47ba-9daf-37826a22ff4a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:57.090645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944730e-320e-40e2-b8a1-78c00f7d102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:57.091296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944730e-320e-40e2-b8a1-78c00f7d102c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:57.091584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944730e-320e-40e2-b8a1-78c00f7d102c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:57.091864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944730e-320e-40e2-b8a1-78c00f7d102c None admin] PATH_INFO: `/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:57.093059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944730e-320e-40e2-b8a1-78c00f7d102c None admin] RBAC: Authorizing `identity:get_user(user_id=f59916fe4d6d4a6aac1fc1ff790ac163)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:57.094557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944730e-320e-40e2-b8a1-78c00f7d102c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:57.096170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 26/53] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:06:56 2022] GET /identity/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163 => generated 248 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:57.101688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944730e-320e-40e2-b8a1-78c00f7d102c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:57.113022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:57.128487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c34d9e-7a00-4de9-8d4f-a4231a6e46a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:57.129139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34d9e-7a00-4de9-8d4f-a4231a6e46a0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:57.129430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34d9e-7a00-4de9-8d4f-a4231a6e46a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:57.129724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34d9e-7a00-4de9-8d4f-a4231a6e46a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:57.130885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c34d9e-7a00-4de9-8d4f-a4231a6e46a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:57.132347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c34d9e-7a00-4de9-8d4f-a4231a6e46a0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:57.134612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 28/54] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:06:56 2022] GET /identity/v3/domains/default => generated 200 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:57.140256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c34d9e-7a00-4de9-8d4f-a4231a6e46a0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:57.204245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944730e-320e-40e2-b8a1-78c00f7d102c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:57.237699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de964cd-6e8a-40ac-8453-2ffe28733289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:57.238337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de964cd-6e8a-40ac-8453-2ffe28733289 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:57.238433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c34d9e-7a00-4de9-8d4f-a4231a6e46a0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:57.238616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de964cd-6e8a-40ac-8453-2ffe28733289 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:57.238888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de964cd-6e8a-40ac-8453-2ffe28733289 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:57.240035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de964cd-6e8a-40ac-8453-2ffe28733289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:57.241517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de964cd-6e8a-40ac-8453-2ffe28733289 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:57.243103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 27/55] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:06:57 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:57.249431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de964cd-6e8a-40ac-8453-2ffe28733289 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:57.269580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:57.275010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11528870-8436-401a-9a43-b4a5c9b6ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:57.275713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11528870-8436-401a-9a43-b4a5c9b6ddea None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:57.275994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11528870-8436-401a-9a43-b4a5c9b6ddea None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:57.276264 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11528870-8436-401a-9a43-b4a5c9b6ddea None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:57.276937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11528870-8436-401a-9a43-b4a5c9b6ddea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:57.278406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11528870-8436-401a-9a43-b4a5c9b6ddea None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:57.304261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 29/56] 10.222.0.109 () {60 vars in 1145 bytes} [Thu Aug 4 22:06:57 2022] POST /identity/v3/projects => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:06:57.329611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:57.353901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de964cd-6e8a-40ac-8453-2ffe28733289 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:57.391186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ddf07-2bc9-450b-9d5d-efef1fd38a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:57.391823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ddf07-2bc9-450b-9d5d-efef1fd38a7e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:57.392110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ddf07-2bc9-450b-9d5d-efef1fd38a7e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:57.392411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ddf07-2bc9-450b-9d5d-efef1fd38a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:57.393203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ddf07-2bc9-450b-9d5d-efef1fd38a7e None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.domain.id=default, user.id=f59916fe4d6d4a6aac1fc1ff790ac163) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:06:57.393604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ddf07-2bc9-450b-9d5d-efef1fd38a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:57.395104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ddf07-2bc9-450b-9d5d-efef1fd38a7e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:57.402128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 28/57] 10.222.0.109 () {56 vars in 1327 bytes} [Thu Aug 4 22:06:57 2022] GET /identity/v3/role_assignments?user.id=f59916fe4d6d4a6aac1fc1ff790ac163&scope.domain.id=default&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 532 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:06:59.331372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84cb3-18fe-4ce4-bf3b-8410aa7b6e47 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:59.331883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84cb3-18fe-4ce4-bf3b-8410aa7b6e47 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:59.331968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84cb3-18fe-4ce4-bf3b-8410aa7b6e47 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:59.333280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 30/58] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:06:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:06:59.338926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dcc6c-4bc4-4139-865e-ed19736e5e20 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:59.339343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dcc6c-4bc4-4139-865e-ed19736e5e20 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:59.339473 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dcc6c-4bc4-4139-865e-ed19736e5e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:59.357548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0c0dcc6c-4bc4-4139-865e-ed19736e5e20 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:06:59.358727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0c0dcc6c-4bc4-4139-865e-ed19736e5e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:06:59.422236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 29/59] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:06:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:06:59.446242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb84cb3-18fe-4ce4-bf3b-8410aa7b6e47 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:59.547549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb84cb3-18fe-4ce4-bf3b-8410aa7b6e47 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:59.579844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7511f-4204-40af-aad9-018dc004b893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:59.580473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7511f-4204-40af-aad9-018dc004b893 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:59.580755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7511f-4204-40af-aad9-018dc004b893 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:59.581028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7511f-4204-40af-aad9-018dc004b893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:59.582196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7511f-4204-40af-aad9-018dc004b893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:59.583637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7511f-4204-40af-aad9-018dc004b893 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:59.585205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 31/60] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:06:59 2022] GET /identity/v3/domains/default => generated 200 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:06:59.591282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dcc6c-4bc4-4139-865e-ed19736e5e20 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:06:59.658725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:06:59.688512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dcc6c-4bc4-4139-865e-ed19736e5e20 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:06:59.721240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bc4f0-af9f-4810-b627-7f1497d2bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:06:59.721882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bc4f0-af9f-4810-b627-7f1497d2bde5 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:06:59.722164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bc4f0-af9f-4810-b627-7f1497d2bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:06:59.722437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bc4f0-af9f-4810-b627-7f1497d2bde5 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:06:59.723127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bc4f0-af9f-4810-b627-7f1497d2bde5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:06:59.724586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bc4f0-af9f-4810-b627-7f1497d2bde5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:06:59.749786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 30/61] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:06:59 2022] POST /identity/v3/users => generated 276 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:02.897960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046f12a-4574-4cbd-a588-89510350cc70 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:02.898493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046f12a-4574-4cbd-a588-89510350cc70 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:02.898597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046f12a-4574-4cbd-a588-89510350cc70 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:02.899958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 32/62] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:02.905527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d688f1-e002-4996-be89-bf6dae7ad8cf None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:02.905954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d688f1-e002-4996-be89-bf6dae7ad8cf None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:02.906163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d688f1-e002-4996-be89-bf6dae7ad8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:02.938103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e1d688f1-e002-4996-be89-bf6dae7ad8cf None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:02.939304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e1d688f1-e002-4996-be89-bf6dae7ad8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:03.031328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 31/63] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:03.072910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046f12a-4574-4cbd-a588-89510350cc70 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.086374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea2bde-32e0-4ed6-ab12-2323b2a6627d None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.087242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea2bde-32e0-4ed6-ab12-2323b2a6627d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.087457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea2bde-32e0-4ed6-ab12-2323b2a6627d None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.088841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 32/64] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:03.094549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc70b7-2db4-4fc7-bf3a-2836d4b13fcb None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.094838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc70b7-2db4-4fc7-bf3a-2836d4b13fcb None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.095121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc70b7-2db4-4fc7-bf3a-2836d4b13fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.115851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9edc70b7-2db4-4fc7-bf3a-2836d4b13fcb None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:03.117193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9edc70b7-2db4-4fc7-bf3a-2836d4b13fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:03.163808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:03.181624 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:03.206010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 33/65] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:03.213429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95563-faed-4a8c-805e-5d0f270f1b02 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.213429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95563-faed-4a8c-805e-5d0f270f1b02 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.213429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95563-faed-4a8c-805e-5d0f270f1b02 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.213429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 34/66] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:03 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:03.218553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afe63-fc9f-4826-831b-01ed4a5cb2e9 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.218846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afe63-fc9f-4826-831b-01ed4a5cb2e9 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.219125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afe63-fc9f-4826-831b-01ed4a5cb2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.238733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046f12a-4574-4cbd-a588-89510350cc70 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:03.248159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-fd7afe63-fc9f-4826-831b-01ed4a5cb2e9 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:03.249369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fd7afe63-fc9f-4826-831b-01ed4a5cb2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:03.283392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0650-8285-4785-a3a9-f84911fa6cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:03.284590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0650-8285-4785-a3a9-f84911fa6cb4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.285074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0650-8285-4785-a3a9-f84911fa6cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.285572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0650-8285-4785-a3a9-f84911fa6cb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.299786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a0650-8285-4785-a3a9-f84911fa6cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-232a0650-8285-4785-a3a9-f84911fa6cb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:03.307082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:03.307975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:03.308728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:03.315798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 33/67] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:03.315798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0650-8285-4785-a3a9-f84911fa6cb4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.332811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 35/68] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:03.334008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7afe63-fc9f-4826-831b-01ed4a5cb2e9 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.427519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0650-8285-4785-a3a9-f84911fa6cb4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:03.438883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7afe63-fc9f-4826-831b-01ed4a5cb2e9 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:03.440378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:03.456690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:03.466445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ed2ab-0604-4f0a-91fc-008210a16153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:03.467099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed2ab-0604-4f0a-91fc-008210a16153 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.467381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed2ab-0604-4f0a-91fc-008210a16153 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.467654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed2ab-0604-4f0a-91fc-008210a16153 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.476840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:03.477517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.477798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.478145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.478249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ed2ab-0604-4f0a-91fc-008210a16153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:03.478871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] RBAC: Adding query filter params (name=member) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:03.479202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:03.480578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-405ed2ab-0604-4f0a-91fc-008210a16153 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:03.483601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:03.484417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:03.485167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 34/69] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:03.490101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 36/70] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:03.490921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.491081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f2f4-8023-48b6-be30-bcdcfc3000a6 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.491350 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f2f4-8023-48b6-be30-bcdcfc3000a6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.491612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f2f4-8023-48b6-be30-bcdcfc3000a6 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.497645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 35/71] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:03 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:03.498134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f2f4-8023-48b6-be30-bcdcfc3000a6 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.596273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685a88a-5c1f-4e28-94e5-36f92b90efff None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:03.604833 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f2f4-8023-48b6-be30-bcdcfc3000a6 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:03.628614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d8669-825c-4dc8-a2f1-8971fceb4fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:03.629255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8669-825c-4dc8-a2f1-8971fceb4fa5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.629574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8669-825c-4dc8-a2f1-8971fceb4fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.629850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8669-825c-4dc8-a2f1-8971fceb4fa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.637992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb5fed-59f4-440c-841a-db629731748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:03.638614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb5fed-59f4-440c-841a-db629731748a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.638900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb5fed-59f4-440c-841a-db629731748a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.639175 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb5fed-59f4-440c-841a-db629731748a None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.639887 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb5fed-59f4-440c-841a-db629731748a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:03.640192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb5fed-59f4-440c-841a-db629731748a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:03.640439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d8669-825c-4dc8-a2f1-8971fceb4fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:03.641606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb5fed-59f4-440c-841a-db629731748a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-093d8669-825c-4dc8-a2f1-8971fceb4fa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:03.646109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:03.647085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 37/72] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:03.648015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d8669-825c-4dc8-a2f1-8971fceb4fa5 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.655309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 36/73] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:03.656602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77a9cc-c809-48c1-90ab-66884f5b2f74 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.656898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77a9cc-c809-48c1-90ab-66884f5b2f74 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.657176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77a9cc-c809-48c1-90ab-66884f5b2f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.675046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-cd77a9cc-c809-48c1-90ab-66884f5b2f74 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:03.676257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cd77a9cc-c809-48c1-90ab-66884f5b2f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:03.689652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:03.703518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:03.743308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 37/74] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:03.744521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30762697-b177-462f-a221-14018961cb65 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.744810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30762697-b177-462f-a221-14018961cb65 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.745079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30762697-b177-462f-a221-14018961cb65 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.749580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 38/75] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:03.749580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30762697-b177-462f-a221-14018961cb65 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.753302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d8669-825c-4dc8-a2f1-8971fceb4fa5 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:03.788003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd0dfb-560b-40e8-a616-b22ac91e8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:03.788643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd0dfb-560b-40e8-a616-b22ac91e8684 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.788931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd0dfb-560b-40e8-a616-b22ac91e8684 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.789201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd0dfb-560b-40e8-a616-b22ac91e8684 None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.802257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd0dfb-560b-40e8-a616-b22ac91e8684 None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:03.803741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd0dfb-560b-40e8-a616-b22ac91e8684 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:03.805421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 38/76] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:03.806500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd0dfb-560b-40e8-a616-b22ac91e8684 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.859996 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30762697-b177-462f-a221-14018961cb65 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:03.897797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886acf9-b717-47dc-b565-01ed4f42eef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:03.898418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886acf9-b717-47dc-b565-01ed4f42eef7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.898944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886acf9-b717-47dc-b565-01ed4f42eef7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.898944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886acf9-b717-47dc-b565-01ed4f42eef7 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.899713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886acf9-b717-47dc-b565-01ed4f42eef7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:03.900014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886acf9-b717-47dc-b565-01ed4f42eef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:03.900730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:03.901639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886acf9-b717-47dc-b565-01ed4f42eef7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:03.906901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd0dfb-560b-40e8-a616-b22ac91e8684 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:03.910479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 39/77] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:03.911664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432236a4-8240-4d30-b741-ea114e0d134b None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.911952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432236a4-8240-4d30-b741-ea114e0d134b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.912220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432236a4-8240-4d30-b741-ea114e0d134b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.929054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-432236a4-8240-4d30-b741-ea114e0d134b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:03.930249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-432236a4-8240-4d30-b741-ea114e0d134b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:03.942197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45180bb-641f-4665-bae2-13eaeb8aff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:03.942782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45180bb-641f-4665-bae2-13eaeb8aff01 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:03.943060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45180bb-641f-4665-bae2-13eaeb8aff01 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:03.943331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45180bb-641f-4665-bae2-13eaeb8aff01 None admin] PATH_INFO: `/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:03.944489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45180bb-641f-4665-bae2-13eaeb8aff01 None admin] RBAC: Authorizing `identity:get_user(user_id=f59916fe4d6d4a6aac1fc1ff790ac163)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:03.945976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45180bb-641f-4665-bae2-13eaeb8aff01 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:03.948316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 39/78] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163 => generated 248 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:03.949077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45180bb-641f-4665-bae2-13eaeb8aff01 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:03.951624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:03.998650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 40/79] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:03.999790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432236a4-8240-4d30-b741-ea114e0d134b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.049651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45180bb-641f-4665-bae2-13eaeb8aff01 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.083487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb126a-a36e-4f98-8fb5-84487da042da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.084128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb126a-a36e-4f98-8fb5-84487da042da None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.084412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb126a-a36e-4f98-8fb5-84487da042da None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.084692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb126a-a36e-4f98-8fb5-84487da042da None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.094962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb126a-a36e-4f98-8fb5-84487da042da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-12bb126a-a36e-4f98-8fb5-84487da042da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:04.099857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:04.100769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:04.107182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:04.107899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432236a4-8240-4d30-b741-ea114e0d134b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.107981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 40/80] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:04.107981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb126a-a36e-4f98-8fb5-84487da042da None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.130770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:04.142027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e95e-3520-43bc-b2b2-b0c6cb53131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.142661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e95e-3520-43bc-b2b2-b0c6cb53131b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.142942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e95e-3520-43bc-b2b2-b0c6cb53131b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.143213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e95e-3520-43bc-b2b2-b0c6cb53131b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.168499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:04.206294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb126a-a36e-4f98-8fb5-84487da042da None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.210666 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e95e-3520-43bc-b2b2-b0c6cb53131b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:04.212724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e95e-3520-43bc-b2b2-b0c6cb53131b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:04.239677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a2de4-1bc0-4ec4-bb73-421490dbfc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.240314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2de4-1bc0-4ec4-bb73-421490dbfc71 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.240619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2de4-1bc0-4ec4-bb73-421490dbfc71 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.240912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2de4-1bc0-4ec4-bb73-421490dbfc71 None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.242107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a2de4-1bc0-4ec4-bb73-421490dbfc71 None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:04.243815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a2de4-1bc0-4ec4-bb73-421490dbfc71 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:04.245426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 41/81] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:04 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:04.246187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a2de4-1bc0-4ec4-bb73-421490dbfc71 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.311110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 41/82] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:03 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:04.312369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e95e-3520-43bc-b2b2-b0c6cb53131b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.336391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:04.346047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a2de4-1bc0-4ec4-bb73-421490dbfc71 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.367279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:04.379957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3c89b-b4d0-4e4b-badd-180356fb5cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.380577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3c89b-b4d0-4e4b-badd-180356fb5cc8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.380867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3c89b-b4d0-4e4b-badd-180356fb5cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.381148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3c89b-b4d0-4e4b-badd-180356fb5cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.411411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e95e-3520-43bc-b2b2-b0c6cb53131b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.444218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a8eba-c987-4ca2-9807-47e530ccb75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.444852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8eba-c987-4ca2-9807-47e530ccb75f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.445130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8eba-c987-4ca2-9807-47e530ccb75f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.445416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8eba-c987-4ca2-9807-47e530ccb75f None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.447185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a8eba-c987-4ca2-9807-47e530ccb75f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:04.448627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a8eba-c987-4ca2-9807-47e530ccb75f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:04.449005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3c89b-b4d0-4e4b-badd-180356fb5cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:04.450231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 42/83] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:04 2022] GET /identity/v3/domains/default => generated 200 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:04.450507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3c89b-b4d0-4e4b-badd-180356fb5cc8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:04.450994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a8eba-c987-4ca2-9807-47e530ccb75f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.549961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a8eba-c987-4ca2-9807-47e530ccb75f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.559461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:04.569751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 42/84] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:04 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:04.570574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3c89b-b4d0-4e4b-badd-180356fb5cc8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.575610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:04.588599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.589238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.589549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.589832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.590547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:04.590853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:04.592235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:04.600561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 43/85] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:04 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:04.601345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.668251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3c89b-b4d0-4e4b-badd-180356fb5cc8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.701876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0d4b3-7735-4d86-9905-b1d5fed844cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.702492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d4b3-7735-4d86-9905-b1d5fed844cd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.702768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d4b3-7735-4d86-9905-b1d5fed844cd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.703039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d4b3-7735-4d86-9905-b1d5fed844cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.705048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5112108-8e7d-4202-9ff6-464c284ee8d9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.737621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b304a-e10c-421e-831a-7d930c48b113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.738257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b304a-e10c-421e-831a-7d930c48b113 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.738543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b304a-e10c-421e-831a-7d930c48b113 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.738821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b304a-e10c-421e-831a-7d930c48b113 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.739990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b304a-e10c-421e-831a-7d930c48b113 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:04.741459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b304a-e10c-421e-831a-7d930c48b113 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:04.744581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 44/86] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:04 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:04.745339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b304a-e10c-421e-831a-7d930c48b113 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.766120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:04.776537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0d4b3-7735-4d86-9905-b1d5fed844cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:04.778062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0d4b3-7735-4d86-9905-b1d5fed844cd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:04.799306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:04.844842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b304a-e10c-421e-831a-7d930c48b113 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:04.877553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176d9d7-83f8-4698-a1a0-91a785f54d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:04.878174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d9d7-83f8-4698-a1a0-91a785f54d3a None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:04.878458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d9d7-83f8-4698-a1a0-91a785f54d3a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:04.878731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d9d7-83f8-4698-a1a0-91a785f54d3a None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:04.879371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 43/87] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:04 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:04.879505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176d9d7-83f8-4698-a1a0-91a785f54d3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:04.880243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0d4b3-7735-4d86-9905-b1d5fed844cd None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.880875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176d9d7-83f8-4698-a1a0-91a785f54d3a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:04.909842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 45/88] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:04 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:04.910669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176d9d7-83f8-4698-a1a0-91a785f54d3a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:04.989126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:05.001826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0d4b3-7735-4d86-9905-b1d5fed844cd None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:05.032538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176d9d7-83f8-4698-a1a0-91a785f54d3a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:05.042909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57588f47-3178-4858-bddc-ad319e514946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:05.043580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57588f47-3178-4858-bddc-ad319e514946 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:05.044261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57588f47-3178-4858-bddc-ad319e514946 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:05.044548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57588f47-3178-4858-bddc-ad319e514946 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:05.045476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:05.045960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57588f47-3178-4858-bddc-ad319e514946 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:05.047634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57588f47-3178-4858-bddc-ad319e514946 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:05.049421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 44/89] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:04 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:05.050264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57588f47-3178-4858-bddc-ad319e514946 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:05.082297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede5f9d-fb2b-47ec-837a-f88384138454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:05.082955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede5f9d-fb2b-47ec-837a-f88384138454 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:05.083233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede5f9d-fb2b-47ec-837a-f88384138454 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:05.083510 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede5f9d-fb2b-47ec-837a-f88384138454 None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:05.084677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede5f9d-fb2b-47ec-837a-f88384138454 None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:05.086181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede5f9d-fb2b-47ec-837a-f88384138454 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:05.088053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 46/90] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:04 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:05.088921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede5f9d-fb2b-47ec-837a-f88384138454 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:05.182920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57588f47-3178-4858-bddc-ad319e514946 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:05.225190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede5f9d-fb2b-47ec-837a-f88384138454 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:05.229107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:05.229768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:05.230054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:05.230329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:05.231129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] RBAC: Adding query filter params (role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=3aa2c7542ee14875b5538c96394941d1, user.id=411770b6026d4746a34e34ea2439c6f8) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:05.231440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:05.232949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:05.238789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 45/91] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:05 2022] GET /identity/v3/role_assignments?user.id=411770b6026d4746a34e34ea2439c6f8&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:05.239605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:05.272089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a039e5-6244-40dd-b20f-14972261e229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:05.272762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a039e5-6244-40dd-b20f-14972261e229 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:05.273039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a039e5-6244-40dd-b20f-14972261e229 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:05.273372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a039e5-6244-40dd-b20f-14972261e229 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:05.275463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a039e5-6244-40dd-b20f-14972261e229 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:05.277042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a039e5-6244-40dd-b20f-14972261e229 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:05.278586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:05.278844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 47/92] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:05 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:05.282425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a039e5-6244-40dd-b20f-14972261e229 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:05.337698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:05.387047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca838b5-8ef6-4652-a283-e6112ea5d6ba None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:05.406431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a039e5-6244-40dd-b20f-14972261e229 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:05.436413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:05.437056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:05.437480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:05.437935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:05.439903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=3aa2c7542ee14875b5538c96394941d1, user.id=f59916fe4d6d4a6aac1fc1ff790ac163) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:05.442144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:05.444785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:05.450265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bb52b-a0b1-4227-9b4e-60d8f76d2cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:05.451224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bb52b-a0b1-4227-9b4e-60d8f76d2cb8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:05.451701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bb52b-a0b1-4227-9b4e-60d8f76d2cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:05.452822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 46/93] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:05 2022] GET /identity/v3/role_assignments?user.id=f59916fe4d6d4a6aac1fc1ff790ac163&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:05.453220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bb52b-a0b1-4227-9b4e-60d8f76d2cb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:05.453524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:05.561448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bb52b-a0b1-4227-9b4e-60d8f76d2cb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:05.563400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bb52b-a0b1-4227-9b4e-60d8f76d2cb8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:05.621320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:05.627521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:05.659736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17fb65-5c04-416b-943a-3fd9bb5a1029 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:05.735698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:05.736336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:05.736618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:05.736892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:05.737865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] RBAC: Adding query filter params (role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=3aa2c7542ee14875b5538c96394941d1, user.id=411770b6026d4746a34e34ea2439c6f8) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:05.738177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:05.739654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:05.753028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 48/94] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:05 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:05.760217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 47/95] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:05 2022] GET /identity/v3/role_assignments?user.id=411770b6026d4746a34e34ea2439c6f8&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=8f997477100248a9ace1cc28831caaac => generated 260 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:05.761671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bb52b-a0b1-4227-9b4e-60d8f76d2cb8 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:05.916310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bb52b-a0b1-4227-9b4e-60d8f76d2cb8 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:05.975208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9181725-bf2c-4d09-84d3-2a81e94219a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:05.975886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181725-bf2c-4d09-84d3-2a81e94219a8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:05.976172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181725-bf2c-4d09-84d3-2a81e94219a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:05.976451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181725-bf2c-4d09-84d3-2a81e94219a8 None admin] PATH_INFO: `/v3/projects/2840a8dcbce742e19ff5b08413bc8b4a` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:05.982282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9181725-bf2c-4d09-84d3-2a81e94219a8 None admin] RBAC: Authorizing `identity:get_project(project_id=2840a8dcbce742e19ff5b08413bc8b4a)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:05.983800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9181725-bf2c-4d09-84d3-2a81e94219a8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:05.985624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 49/96] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:05 2022] GET /identity/v3/projects/2840a8dcbce742e19ff5b08413bc8b4a => generated 313 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:05.991756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:06.209201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d043cb-07ac-4005-9c12-8f052acb72ab None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:06.244378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de569b28-49f1-46c7-b4cf-af87b2d80225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:06.245008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de569b28-49f1-46c7-b4cf-af87b2d80225 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:06.245290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de569b28-49f1-46c7-b4cf-af87b2d80225 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:06.245605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de569b28-49f1-46c7-b4cf-af87b2d80225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:06.246408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de569b28-49f1-46c7-b4cf-af87b2d80225 None admin] RBAC: Adding query filter params (role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=2840a8dcbce742e19ff5b08413bc8b4a, user.id=411770b6026d4746a34e34ea2439c6f8) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:06.246721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de569b28-49f1-46c7-b4cf-af87b2d80225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:06.248253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de569b28-49f1-46c7-b4cf-af87b2d80225 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:06.254134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 48/97] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:05 2022] GET /identity/v3/role_assignments?user.id=411770b6026d4746a34e34ea2439c6f8&scope.project.id=2840a8dcbce742e19ff5b08413bc8b4a&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:07.893873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b532-0d50-44a8-8b00-713de127c6f4 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:07.894386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b532-0d50-44a8-8b00-713de127c6f4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:07.894679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b532-0d50-44a8-8b00-713de127c6f4 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:07.896267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 50/98] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:07.903099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214460f-5b3e-469a-b8fb-af08cafb0614 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:07.903523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214460f-5b3e-469a-b8fb-af08cafb0614 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:07.903710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214460f-5b3e-469a-b8fb-af08cafb0614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:07.931152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-1214460f-5b3e-469a-b8fb-af08cafb0614 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:07.932298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1214460f-5b3e-469a-b8fb-af08cafb0614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:07.947678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:07.971656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82c699-37e5-44de-910d-9f2db096f8de None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:07.972290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82c699-37e5-44de-910d-9f2db096f8de None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:07.972875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82c699-37e5-44de-910d-9f2db096f8de None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:07.974462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 51/99] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:07.980854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd219-fbbd-46db-8e92-acec4d5ba086 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:07.981389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd219-fbbd-46db-8e92-acec4d5ba086 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:07.981900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd219-fbbd-46db-8e92-acec4d5ba086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.001547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3e1dd219-fbbd-46db-8e92-acec4d5ba086 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:08.003178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3e1dd219-fbbd-46db-8e92-acec4d5ba086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:08.017745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:08.036326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 49/100] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:08.077761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214460f-5b3e-469a-b8fb-af08cafb0614 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:08.115545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 52/101] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:08.117315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18d77-e6e7-4fda-b220-e1b5d7244c37 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.121943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18d77-e6e7-4fda-b220-e1b5d7244c37 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.122444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18d77-e6e7-4fda-b220-e1b5d7244c37 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.123942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 53/102] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:08 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:08.129464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ed48f-c348-4bb0-9ed0-e36ac374a229 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.129989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ed48f-c348-4bb0-9ed0-e36ac374a229 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.130458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ed48f-c348-4bb0-9ed0-e36ac374a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.162511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-222ed48f-c348-4bb0-9ed0-e36ac374a229 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:08.163642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-222ed48f-c348-4bb0-9ed0-e36ac374a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:08.219861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214460f-5b3e-469a-b8fb-af08cafb0614 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:08.260616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eb567-a66f-4e05-8754-1307ace33329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:08.261283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eb567-a66f-4e05-8754-1307ace33329 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.261600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eb567-a66f-4e05-8754-1307ace33329 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.261886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eb567-a66f-4e05-8754-1307ace33329 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.263234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8eb567-a66f-4e05-8754-1307ace33329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:08.264723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8eb567-a66f-4e05-8754-1307ace33329 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:08.267093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 50/103] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:08 2022] GET /identity/v3/domains/default => generated 200 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:08.268166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eb567-a66f-4e05-8754-1307ace33329 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:08.290352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:08.377231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eb567-a66f-4e05-8754-1307ace33329 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:08.417024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 54/104] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:08.417822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ed48f-c348-4bb0-9ed0-e36ac374a229 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:08.419718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af158515-3232-4900-8253-0b88dcd9e833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:08.420370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158515-3232-4900-8253-0b88dcd9e833 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.420729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158515-3232-4900-8253-0b88dcd9e833 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.420980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158515-3232-4900-8253-0b88dcd9e833 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.443907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af158515-3232-4900-8253-0b88dcd9e833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-af158515-3232-4900-8253-0b88dcd9e833 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:08.449487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:08.450317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:08.451481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 51/105] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:08.462202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af158515-3232-4900-8253-0b88dcd9e833 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:08.569815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:08.614903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:08.625134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ed48f-c348-4bb0-9ed0-e36ac374a229 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:08.646981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af158515-3232-4900-8253-0b88dcd9e833 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:08.667890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f67c16-531e-4bc2-a2ae-06240c161bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:08.668445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f67c16-531e-4bc2-a2ae-06240c161bbb None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.668728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f67c16-531e-4bc2-a2ae-06240c161bbb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.669463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f67c16-531e-4bc2-a2ae-06240c161bbb None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.670519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f67c16-531e-4bc2-a2ae-06240c161bbb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:08.672167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f67c16-531e-4bc2-a2ae-06240c161bbb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:08.684141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:08.684855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.685235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.685616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.686409 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:08.686785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:08.688355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:08.701218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 52/106] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:08 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:08.702014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:08.704716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 55/107] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:07:08 2022] POST /identity/v3/projects => generated 303 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:08.706014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb52836-9864-4a5a-98c4-6cb2f034e164 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.706390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb52836-9864-4a5a-98c4-6cb2f034e164 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.706845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb52836-9864-4a5a-98c4-6cb2f034e164 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.707828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 56/108] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:08.708570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb52836-9864-4a5a-98c4-6cb2f034e164 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:08.842003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb52836-9864-4a5a-98c4-6cb2f034e164 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:08.892155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70e888-7bef-4440-9100-43b191f1176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:08.892973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70e888-7bef-4440-9100-43b191f1176b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.893258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70e888-7bef-4440-9100-43b191f1176b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.893576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70e888-7bef-4440-9100-43b191f1176b None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.895382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70e888-7bef-4440-9100-43b191f1176b None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:08.897634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70e888-7bef-4440-9100-43b191f1176b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:08.904849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 57/109] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:08 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:08.904849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd087fb-6733-4b7c-8a47-65d605a599df None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.904849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd087fb-6733-4b7c-8a47-65d605a599df None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.904849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd087fb-6733-4b7c-8a47-65d605a599df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.912873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8eb04-1947-407f-85dc-5da10f1895c9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:08.921883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:08.925462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-dcd087fb-6733-4b7c-8a47-65d605a599df None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:08.927703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-dcd087fb-6733-4b7c-8a47-65d605a599df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:08.959305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738b0f8-c14a-4957-a4dd-fd898b32cd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:08.962909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b0f8-c14a-4957-a4dd-fd898b32cd67 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.963199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b0f8-c14a-4957-a4dd-fd898b32cd67 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.963485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b0f8-c14a-4957-a4dd-fd898b32cd67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.968324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:08.979210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738b0f8-c14a-4957-a4dd-fd898b32cd67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-7738b0f8-c14a-4957-a4dd-fd898b32cd67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:08.984805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:08.985790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:08.986566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:08.987146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 53/110] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:08.988093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04ba58-869d-4b1a-a100-991f6daaa1ab None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:08.988381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04ba58-869d-4b1a-a100-991f6daaa1ab None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:08.988674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04ba58-869d-4b1a-a100-991f6daaa1ab None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:08.991266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 54/111] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:08.991266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04ba58-869d-4b1a-a100-991f6daaa1ab None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.032290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 58/112] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:09.033521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd087fb-6733-4b7c-8a47-65d605a599df None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.136308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04ba58-869d-4b1a-a100-991f6daaa1ab None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:09.163370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd087fb-6733-4b7c-8a47-65d605a599df None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:09.177652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad872be2-9ee1-4e38-b6d7-6e8f1538ef85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:09.178567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872be2-9ee1-4e38-b6d7-6e8f1538ef85 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.178989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872be2-9ee1-4e38-b6d7-6e8f1538ef85 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.179458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872be2-9ee1-4e38-b6d7-6e8f1538ef85 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.181318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad872be2-9ee1-4e38-b6d7-6e8f1538ef85 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:09.183057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad872be2-9ee1-4e38-b6d7-6e8f1538ef85 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:09.184949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 55/113] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:08 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:09.186409 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2076-44e6-4e37-b95a-d1e606ae4a7b None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.186849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2076-44e6-4e37-b95a-d1e606ae4a7b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.187333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2076-44e6-4e37-b95a-d1e606ae4a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.198064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:09.208468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7b0e2076-44e6-4e37-b95a-d1e606ae4a7b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:09.209862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7b0e2076-44e6-4e37-b95a-d1e606ae4a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:09.211417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:09.212258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.212722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.213203 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.214026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:09.214403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:09.215871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:09.225050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 59/114] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:09 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:09.226051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.278905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:09.288111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 56/115] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:09.288829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e2076-44e6-4e37-b95a-d1e606ae4a7b None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.345148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949d0b5-e8a2-44b2-9c93-932f336eab0d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:09.384719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de852a57-9800-48b9-b264-6c63d4a72d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:09.385372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de852a57-9800-48b9-b264-6c63d4a72d69 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.385920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de852a57-9800-48b9-b264-6c63d4a72d69 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.386207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de852a57-9800-48b9-b264-6c63d4a72d69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.402155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de852a57-9800-48b9-b264-6c63d4a72d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:09.402428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e2076-44e6-4e37-b95a-d1e606ae4a7b None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-de852a57-9800-48b9-b264-6c63d4a72d69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:09.408714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:09.409613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:09.410367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:09.410928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 60/116] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:09.417450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9472c34-4963-4d58-9942-1d73954c1d3d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.417450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9472c34-4963-4d58-9942-1d73954c1d3d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.417450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9472c34-4963-4d58-9942-1d73954c1d3d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.417450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 61/117] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:09.417450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9472c34-4963-4d58-9942-1d73954c1d3d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.442506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca080f5f-3fba-4afd-96b8-1e85f46eaea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:09.443117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca080f5f-3fba-4afd-96b8-1e85f46eaea7 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.443399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca080f5f-3fba-4afd-96b8-1e85f46eaea7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.443676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca080f5f-3fba-4afd-96b8-1e85f46eaea7 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1/users/411770b6026d4746a34e34ea2439c6f8/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.445677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca080f5f-3fba-4afd-96b8-1e85f46eaea7 None admin] RBAC: Authorizing `identity:create_grant(project_id=3aa2c7542ee14875b5538c96394941d1, user_id=411770b6026d4746a34e34ea2439c6f8, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:09.447707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca080f5f-3fba-4afd-96b8-1e85f46eaea7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:09.469453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 57/118] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:09 2022] PUT /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1/users/411770b6026d4746a34e34ea2439c6f8/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:09.469453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca080f5f-3fba-4afd-96b8-1e85f46eaea7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.484849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:09.544700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:09.570441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9472c34-4963-4d58-9942-1d73954c1d3d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:09.602942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca080f5f-3fba-4afd-96b8-1e85f46eaea7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:09.626913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be749dd-52c8-46c8-8d7e-46be3449fbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:09.628457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be749dd-52c8-46c8-8d7e-46be3449fbeb None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.628742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be749dd-52c8-46c8-8d7e-46be3449fbeb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.629024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be749dd-52c8-46c8-8d7e-46be3449fbeb None admin] PATH_INFO: `/v3/users/3c46f5d361e8434bb7b3b04d47b8bf8c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.647983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be749dd-52c8-46c8-8d7e-46be3449fbeb None admin] RBAC: Authorizing `identity:get_user(user_id=3c46f5d361e8434bb7b3b04d47b8bf8c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:09.649670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be749dd-52c8-46c8-8d7e-46be3449fbeb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:09.651591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 62/119] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:09 2022] GET /identity/v3/users/3c46f5d361e8434bb7b3b04d47b8bf8c => generated 290 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:09.652179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be749dd-52c8-46c8-8d7e-46be3449fbeb None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.653689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86069a1-0c53-4da1-bb45-975fe968db13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:09.654500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86069a1-0c53-4da1-bb45-975fe968db13 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.654803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86069a1-0c53-4da1-bb45-975fe968db13 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.655183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86069a1-0c53-4da1-bb45-975fe968db13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.670287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86069a1-0c53-4da1-bb45-975fe968db13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a86069a1-0c53-4da1-bb45-975fe968db13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:09.676065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:09.676883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:09.677652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 58/120] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:09.678270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727255bd-0d70-40d9-8e65-4f0233192436 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.678887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727255bd-0d70-40d9-8e65-4f0233192436 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.678887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727255bd-0d70-40d9-8e65-4f0233192436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.698770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-727255bd-0d70-40d9-8e65-4f0233192436 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:09.699887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-727255bd-0d70-40d9-8e65-4f0233192436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:09.776131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 59/121] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:09.777116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727255bd-0d70-40d9-8e65-4f0233192436 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.793734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be749dd-52c8-46c8-8d7e-46be3449fbeb None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:09.797121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:09.832144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:09.837792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:09.838660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.839110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.839547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.840428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:09.840896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:09.842490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:09.851427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 63/122] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:09 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:09.856730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:09.898476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727255bd-0d70-40d9-8e65-4f0233192436 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:09.944716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4888464-2db7-42a4-a66c-bf50eecd4738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:09.945547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888464-2db7-42a4-a66c-bf50eecd4738 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:09.947385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888464-2db7-42a4-a66c-bf50eecd4738 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:09.947816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888464-2db7-42a4-a66c-bf50eecd4738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:09.969911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240835f-cc7f-45fe-a2c9-ef287336cdf1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.009332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.010129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.010423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.010702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.011534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:10.011951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.013355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.022789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 64/123] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:09 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.023618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.036024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4888464-2db7-42a4-a66c-bf50eecd4738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.037769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4888464-2db7-42a4-a66c-bf50eecd4738 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.056335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:10.092833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:10.138089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455121c-aac0-4c05-992c-57309926d054 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.169216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 60/124] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:10.170610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4888464-2db7-42a4-a66c-bf50eecd4738 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.179948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da1241-e1cc-41c8-93eb-9c73c9849fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.180713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da1241-e1cc-41c8-93eb-9c73c9849fe1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.180998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da1241-e1cc-41c8-93eb-9c73c9849fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.181277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da1241-e1cc-41c8-93eb-9c73c9849fe1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.192723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da1241-e1cc-41c8-93eb-9c73c9849fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-b4da1241-e1cc-41c8-93eb-9c73c9849fe1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:10.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:10.200322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 65/125] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:10.201322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da1241-e1cc-41c8-93eb-9c73c9849fe1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.284360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4888464-2db7-42a4-a66c-bf50eecd4738 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.302051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:10.316707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da1241-e1cc-41c8-93eb-9c73c9849fe1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.325328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5233325-68ee-4fde-b40b-c2f53b7345ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.326185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233325-68ee-4fde-b40b-c2f53b7345ca None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.326617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233325-68ee-4fde-b40b-c2f53b7345ca None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.327143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233325-68ee-4fde-b40b-c2f53b7345ca None admin] PATH_INFO: `/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.328576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5233325-68ee-4fde-b40b-c2f53b7345ca None admin] RBAC: Authorizing `identity:get_user(user_id=f59916fe4d6d4a6aac1fc1ff790ac163)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.330436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5233325-68ee-4fde-b40b-c2f53b7345ca None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.332548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 61/126] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163 => generated 248 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.333744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5233325-68ee-4fde-b40b-c2f53b7345ca None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.350530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:10.356329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41ad06-e352-4b5d-8868-002b8842e35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.357041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41ad06-e352-4b5d-8868-002b8842e35d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.357389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41ad06-e352-4b5d-8868-002b8842e35d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.357699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41ad06-e352-4b5d-8868-002b8842e35d None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.358897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41ad06-e352-4b5d-8868-002b8842e35d None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.360591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41ad06-e352-4b5d-8868-002b8842e35d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.362309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 66/127] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.363206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41ad06-e352-4b5d-8868-002b8842e35d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.447434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5233325-68ee-4fde-b40b-c2f53b7345ca None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.475333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41ad06-e352-4b5d-8868-002b8842e35d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.485469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145c02-ce7f-484d-9d30-8a4c1eee91a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.486471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145c02-ce7f-484d-9d30-8a4c1eee91a6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.487083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145c02-ce7f-484d-9d30-8a4c1eee91a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.487553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145c02-ce7f-484d-9d30-8a4c1eee91a6 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.488915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b145c02-ce7f-484d-9d30-8a4c1eee91a6 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.490736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b145c02-ce7f-484d-9d30-8a4c1eee91a6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.492776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 62/128] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.493916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145c02-ce7f-484d-9d30-8a4c1eee91a6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.517664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.518317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.518580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.518917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.519642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:10.519948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.522006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.536716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 67/129] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.537619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.558529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:10.585095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:10.607590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145c02-ce7f-484d-9d30-8a4c1eee91a6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.647319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11677-9baa-40d9-9b7c-d4589872237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.648264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11677-9baa-40d9-9b7c-d4589872237b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.648812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11677-9baa-40d9-9b7c-d4589872237b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.649327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11677-9baa-40d9-9b7c-d4589872237b None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.650850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11677-9baa-40d9-9b7c-d4589872237b None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.652500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11677-9baa-40d9-9b7c-d4589872237b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.654646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 63/130] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.655837 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11677-9baa-40d9-9b7c-d4589872237b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.663171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2855c-5998-4590-96c3-cc30d8323065 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.704318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee88b3-0212-4a97-ae1d-c03838b26cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.705023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee88b3-0212-4a97-ae1d-c03838b26cda None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.705307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee88b3-0212-4a97-ae1d-c03838b26cda None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.705656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee88b3-0212-4a97-ae1d-c03838b26cda None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.706936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee88b3-0212-4a97-ae1d-c03838b26cda None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.708499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee88b3-0212-4a97-ae1d-c03838b26cda None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.710567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 68/131] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.712569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee88b3-0212-4a97-ae1d-c03838b26cda None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.771483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11677-9baa-40d9-9b7c-d4589872237b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.809399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:10.811250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.812444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.812940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.813507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.814699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] RBAC: Adding query filter params (role.id=b806354bf1c840cb9329d2e65a7576f2, scope.project.id=3aa2c7542ee14875b5538c96394941d1, user.id=3c46f5d361e8434bb7b3b04d47b8bf8c) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:10.815215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.816869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.824207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 64/132] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/role_assignments?user.id=3c46f5d361e8434bb7b3b04d47b8bf8c&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=b806354bf1c840cb9329d2e65a7576f2 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.825317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.832234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee88b3-0212-4a97-ae1d-c03838b26cda None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:10.846932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:10.888750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1579bc6-543b-4a8a-a36c-14d78606dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:10.889435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579bc6-543b-4a8a-a36c-14d78606dfa1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:10.889727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579bc6-543b-4a8a-a36c-14d78606dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:10.890005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579bc6-543b-4a8a-a36c-14d78606dfa1 None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:10.891351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1579bc6-543b-4a8a-a36c-14d78606dfa1 None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:10.892934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1579bc6-543b-4a8a-a36c-14d78606dfa1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:10.894810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 69/133] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:10 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:10.897163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1579bc6-543b-4a8a-a36c-14d78606dfa1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:10.990519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0158-d6c4-43a7-9641-b4057555e60a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:11.020417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1579bc6-543b-4a8a-a36c-14d78606dfa1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:11.030947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2ef57-5594-4e94-b821-17272ba44aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:11.032003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2ef57-5594-4e94-b821-17272ba44aca None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.032614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2ef57-5594-4e94-b821-17272ba44aca None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.033118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2ef57-5594-4e94-b821-17272ba44aca None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1/users/f59916fe4d6d4a6aac1fc1ff790ac163/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.035618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2ef57-5594-4e94-b821-17272ba44aca None admin] RBAC: Authorizing `identity:create_grant(project_id=3aa2c7542ee14875b5538c96394941d1, user_id=f59916fe4d6d4a6aac1fc1ff790ac163, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:11.037567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2ef57-5594-4e94-b821-17272ba44aca None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:11.057910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 65/134] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:10 2022] PUT /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1/users/f59916fe4d6d4a6aac1fc1ff790ac163/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:11.059351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2ef57-5594-4e94-b821-17272ba44aca None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:11.062117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19d9de-21b2-46c2-bfee-5f153c14d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:11.063051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d9de-21b2-46c2-bfee-5f153c14d8fb None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.063281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d9de-21b2-46c2-bfee-5f153c14d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.063659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d9de-21b2-46c2-bfee-5f153c14d8fb None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1/users/411770b6026d4746a34e34ea2439c6f8/roles/8f997477100248a9ace1cc28831caaac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.066195 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19d9de-21b2-46c2-bfee-5f153c14d8fb None admin] RBAC: Authorizing `identity:create_grant(project_id=3aa2c7542ee14875b5538c96394941d1, user_id=411770b6026d4746a34e34ea2439c6f8, role_id=8f997477100248a9ace1cc28831caaac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:11.067901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19d9de-21b2-46c2-bfee-5f153c14d8fb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:11.096450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 70/135] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:10 2022] PUT /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1/users/411770b6026d4746a34e34ea2439c6f8/roles/8f997477100248a9ace1cc28831caaac => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:11.097941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dc602-4481-494d-925e-886f750e3551 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.098311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dc602-4481-494d-925e-886f750e3551 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.098502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dc602-4481-494d-925e-886f750e3551 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.105066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 71/136] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:11 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:11.114493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9701a-7363-4507-95e2-e29f1fca6a9f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.114783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9701a-7363-4507-95e2-e29f1fca6a9f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.114979 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9701a-7363-4507-95e2-e29f1fca6a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.158583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-58c9701a-7363-4507-95e2-e29f1fca6a9f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:11.159900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-58c9701a-7363-4507-95e2-e29f1fca6a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:11.173940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:11.181171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:11.216086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2ef57-5594-4e94-b821-17272ba44aca None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:11.257353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd5bb9-d382-4c18-86cc-f3d29c43759e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:11.259065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd5bb9-d382-4c18-86cc-f3d29c43759e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.259676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd5bb9-d382-4c18-86cc-f3d29c43759e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.260191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd5bb9-d382-4c18-86cc-f3d29c43759e None admin] PATH_INFO: `/v3/projects/2840a8dcbce742e19ff5b08413bc8b4a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.261870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd5bb9-d382-4c18-86cc-f3d29c43759e None admin] RBAC: Authorizing `identity:get_project(project_id=2840a8dcbce742e19ff5b08413bc8b4a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:11.263702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd5bb9-d382-4c18-86cc-f3d29c43759e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:11.265860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 66/137] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:11 2022] GET /identity/v3/projects/2840a8dcbce742e19ff5b08413bc8b4a => generated 313 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:11.275706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd5bb9-d382-4c18-86cc-f3d29c43759e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:11.337010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 72/138] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:11.394727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9701a-7363-4507-95e2-e29f1fca6a9f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:11.443840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd5bb9-d382-4c18-86cc-f3d29c43759e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:11.542694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9701a-7363-4507-95e2-e29f1fca6a9f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:11.546131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f711d4-2a01-45af-8541-911f2224d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:11.546838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f711d4-2a01-45af-8541-911f2224d109 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.547124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f711d4-2a01-45af-8541-911f2224d109 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.547403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f711d4-2a01-45af-8541-911f2224d109 None admin] PATH_INFO: `/v3/projects/2840a8dcbce742e19ff5b08413bc8b4a/users/411770b6026d4746a34e34ea2439c6f8/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.549485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f711d4-2a01-45af-8541-911f2224d109 None admin] RBAC: Authorizing `identity:create_grant(project_id=2840a8dcbce742e19ff5b08413bc8b4a, user_id=411770b6026d4746a34e34ea2439c6f8, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:11.550989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f711d4-2a01-45af-8541-911f2224d109 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:11.574755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:11.584814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 67/139] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:11 2022] PUT /identity/v3/projects/2840a8dcbce742e19ff5b08413bc8b4a/users/411770b6026d4746a34e34ea2439c6f8/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:11.585436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c9f3c-fb21-4807-b9fa-eddf62ed6de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:11.586114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c9f3c-fb21-4807-b9fa-eddf62ed6de7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.586543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c9f3c-fb21-4807-b9fa-eddf62ed6de7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.586827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c9f3c-fb21-4807-b9fa-eddf62ed6de7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.588086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c9f3c-fb21-4807-b9fa-eddf62ed6de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:11.589628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c9f3c-fb21-4807-b9fa-eddf62ed6de7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:11.591404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 73/140] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:11 2022] GET /identity/v3/domains/default => generated 200 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:11.597476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f711d4-2a01-45af-8541-911f2224d109 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:11.711558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f711d4-2a01-45af-8541-911f2224d109 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:11.749780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad2a89-c9db-4cd4-b63f-763f777ee879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:11.750424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2a89-c9db-4cd4-b63f-763f777ee879 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.750707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2a89-c9db-4cd4-b63f-763f777ee879 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.750979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2a89-c9db-4cd4-b63f-763f777ee879 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.751774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad2a89-c9db-4cd4-b63f-763f777ee879 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:11.753270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad2a89-c9db-4cd4-b63f-763f777ee879 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:11.773142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b950da-6cab-4480-a0f3-0525967a5cf4 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.773530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b950da-6cab-4480-a0f3-0525967a5cf4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.774507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b950da-6cab-4480-a0f3-0525967a5cf4 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.776180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 74/141] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:11.781933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9968534-4aad-48c2-b02e-16ba8b5bf954 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:11.782440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9968534-4aad-48c2-b02e-16ba8b5bf954 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:11.782730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9968534-4aad-48c2-b02e-16ba8b5bf954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:11.794139 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 68/142] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:11 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:11.806810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f9968534-4aad-48c2-b02e-16ba8b5bf954 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:11.808110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f9968534-4aad-48c2-b02e-16ba8b5bf954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:11.818158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:11.910279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 75/143] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:11.948575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad2a89-c9db-4cd4-b63f-763f777ee879 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:12.031428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:12.080981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad2a89-c9db-4cd4-b63f-763f777ee879 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:12.126123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f089b74-c803-46fb-a2b9-ce3299b624f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:12.127003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f089b74-c803-46fb-a2b9-ce3299b624f7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:12.127426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f089b74-c803-46fb-a2b9-ce3299b624f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:12.127862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f089b74-c803-46fb-a2b9-ce3299b624f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:12.139149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f089b74-c803-46fb-a2b9-ce3299b624f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-7f089b74-c803-46fb-a2b9-ce3299b624f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:12.145456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:12.146517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:12.147565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:12.150472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 69/144] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:12.156397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9968534-4aad-48c2-b02e-16ba8b5bf954 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:12.296221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9968534-4aad-48c2-b02e-16ba8b5bf954 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:12.334936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:12.335574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:12.335924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:12.336710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:12.337619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:12.337935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:12.339340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:12.348564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 76/145] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:12 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:12.354664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f089b74-c803-46fb-a2b9-ce3299b624f7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:12.494101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f089b74-c803-46fb-a2b9-ce3299b624f7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:12.516946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:12.537317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda3d27-168f-4754-bf9f-f198a34a048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:12.538252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3d27-168f-4754-bf9f-f198a34a048d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:12.538682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3d27-168f-4754-bf9f-f198a34a048d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:12.539096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3d27-168f-4754-bf9f-f198a34a048d None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:12.541485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda3d27-168f-4754-bf9f-f198a34a048d None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:12.543226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda3d27-168f-4754-bf9f-f198a34a048d None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:12.545126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 70/146] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:12 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:12.551507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:12.562788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:12.681531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b516c-57c9-47ab-9af8-7ddce398011a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:12.719722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbab6f-3890-4ef4-a7cf-027d90d0af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:12.720353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbab6f-3890-4ef4-a7cf-027d90d0af18 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:12.720635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbab6f-3890-4ef4-a7cf-027d90d0af18 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:12.720910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbab6f-3890-4ef4-a7cf-027d90d0af18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:12.808553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbab6f-3890-4ef4-a7cf-027d90d0af18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:12.809993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbab6f-3890-4ef4-a7cf-027d90d0af18 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:12.813319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:12.932621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 77/147] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:12 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:12.938804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda3d27-168f-4754-bf9f-f198a34a048d None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:13.077309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda3d27-168f-4754-bf9f-f198a34a048d None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:13.123529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e6422-8d23-45bf-a87a-cc0e0223b292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:13.124191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6422-8d23-45bf-a87a-cc0e0223b292 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:13.124475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6422-8d23-45bf-a87a-cc0e0223b292 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:13.124758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6422-8d23-45bf-a87a-cc0e0223b292 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:13.131370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e6422-8d23-45bf-a87a-cc0e0223b292 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:13.133275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e6422-8d23-45bf-a87a-cc0e0223b292 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:13.136340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 71/148] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:12 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:13.142828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbab6f-3890-4ef4-a7cf-027d90d0af18 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:13.271916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbab6f-3890-4ef4-a7cf-027d90d0af18 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:13.285785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:13.309334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0163e-9237-4371-adef-23c9eb355490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:13.310001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0163e-9237-4371-adef-23c9eb355490 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:13.310277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0163e-9237-4371-adef-23c9eb355490 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:13.310542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0163e-9237-4371-adef-23c9eb355490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:13.311326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0163e-9237-4371-adef-23c9eb355490 None admin] RBAC: Adding query filter params (role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=3aa2c7542ee14875b5538c96394941d1, user.id=411770b6026d4746a34e34ea2439c6f8) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:13.311625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0163e-9237-4371-adef-23c9eb355490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:13.313061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0163e-9237-4371-adef-23c9eb355490 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:13.322053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 78/149] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:13 2022] GET /identity/v3/role_assignments?user.id=411770b6026d4746a34e34ea2439c6f8&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 610 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:14.254786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69e120-3b05-48c8-b0c7-52fcc3085ffb None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:14.255319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69e120-3b05-48c8-b0c7-52fcc3085ffb None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:14.255319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69e120-3b05-48c8-b0c7-52fcc3085ffb None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:14.256862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 72/150] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:14.271619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88072632-ba42-413c-8dca-d1176f9be6e4 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:14.272179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88072632-ba42-413c-8dca-d1176f9be6e4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:14.273069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88072632-ba42-413c-8dca-d1176f9be6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:14.298412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-88072632-ba42-413c-8dca-d1176f9be6e4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:14.300009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-88072632-ba42-413c-8dca-d1176f9be6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:14.390741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 79/151] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:14.420385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69e120-3b05-48c8-b0c7-52fcc3085ffb None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:14.490369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:14.551284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69e120-3b05-48c8-b0c7-52fcc3085ffb None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:14.597371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab9f19-3739-4e33-ba86-094f6123c200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:14.598068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab9f19-3739-4e33-ba86-094f6123c200 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:14.598353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab9f19-3739-4e33-ba86-094f6123c200 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:14.598637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab9f19-3739-4e33-ba86-094f6123c200 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:14.602355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451a922-a277-4ddc-839e-c35bfe785c73 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:14.602355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451a922-a277-4ddc-839e-c35bfe785c73 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:14.602355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451a922-a277-4ddc-839e-c35bfe785c73 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:14.604412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 80/152] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:14.615801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc70f27-a16f-4799-9f50-4f21286e86f3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:14.616371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc70f27-a16f-4799-9f50-4f21286e86f3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:14.617048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc70f27-a16f-4799-9f50-4f21286e86f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:14.620276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab9f19-3739-4e33-ba86-094f6123c200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-5fab9f19-3739-4e33-ba86-094f6123c200 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:14.633945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:14.635014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:14.635802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 73/153] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:14.659503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab9f19-3739-4e33-ba86-094f6123c200 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:14.659665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ffc70f27-a16f-4799-9f50-4f21286e86f3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:14.661165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ffc70f27-a16f-4799-9f50-4f21286e86f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:14.761053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 81/154] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:14.812208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc70f27-a16f-4799-9f50-4f21286e86f3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:14.858143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:14.884248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab9f19-3739-4e33-ba86-094f6123c200 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:14.913681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:14.944941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:14.946822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:14.947323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:14.947863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:14.948777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] RBAC: Adding query filter params (name=reader) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:14.949285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:14.951500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:14.965312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 74/155] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:14 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:14.971204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:15.061457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc70f27-a16f-4799-9f50-4f21286e86f3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:15.102028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661daee2-4a42-48ab-b35c-d8eef693314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:15.107243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661daee2-4a42-48ab-b35c-d8eef693314d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.107243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661daee2-4a42-48ab-b35c-d8eef693314d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.107243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661daee2-4a42-48ab-b35c-d8eef693314d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.118160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661daee2-4a42-48ab-b35c-d8eef693314d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-661daee2-4a42-48ab-b35c-d8eef693314d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:15.123824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:15.124872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:15.125750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 82/156] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:15.127106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5170c0a-088a-49f1-bbec-b0abf99e3be1 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.127106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5170c0a-088a-49f1-bbec-b0abf99e3be1 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.127282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5170c0a-088a-49f1-bbec-b0abf99e3be1 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.128471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 83/157] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:15.134520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5170c0a-088a-49f1-bbec-b0abf99e3be1 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:15.157069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f8da1-6db0-4f1d-8a80-5f3ac28656db None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:15.265567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5170c0a-088a-49f1-bbec-b0abf99e3be1 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:15.274316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:15.281274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56ef4-711b-4b59-ae84-e54313c865ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:15.282607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56ef4-711b-4b59-ae84-e54313c865ce None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.283093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56ef4-711b-4b59-ae84-e54313c865ce None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.283560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56ef4-711b-4b59-ae84-e54313c865ce None admin] PATH_INFO: `/v3/users/3c46f5d361e8434bb7b3b04d47b8bf8c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.285064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f56ef4-711b-4b59-ae84-e54313c865ce None admin] RBAC: Authorizing `identity:get_user(user_id=3c46f5d361e8434bb7b3b04d47b8bf8c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:15.286851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f56ef4-711b-4b59-ae84-e54313c865ce None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:15.290006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 75/158] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:14 2022] GET /identity/v3/users/3c46f5d361e8434bb7b3b04d47b8bf8c => generated 290 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:15.295932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067646dd-279e-414f-8565-7883ff0701b6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.296572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067646dd-279e-414f-8565-7883ff0701b6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.297066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067646dd-279e-414f-8565-7883ff0701b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.326139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:15.326785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.327070 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.327349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.328074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:15.328377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:15.333914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:15.337191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-067646dd-279e-414f-8565-7883ff0701b6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:15.338467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-067646dd-279e-414f-8565-7883ff0701b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:15.362559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 84/159] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:15 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:15.363251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:15.432079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:15.510897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 76/160] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:15.511852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067646dd-279e-414f-8565-7883ff0701b6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:15.551277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61772820-b892-450d-857a-f1529b2d7091 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:15.588961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95eb8a-047c-40ed-896b-7b47b2a09cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:15.589640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95eb8a-047c-40ed-896b-7b47b2a09cc0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.589922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95eb8a-047c-40ed-896b-7b47b2a09cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.590209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95eb8a-047c-40ed-896b-7b47b2a09cc0 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.591376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95eb8a-047c-40ed-896b-7b47b2a09cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:15.593705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95eb8a-047c-40ed-896b-7b47b2a09cc0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:15.595399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 85/161] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:15 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:15.596163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95eb8a-047c-40ed-896b-7b47b2a09cc0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:15.618838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:15.710354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95eb8a-047c-40ed-896b-7b47b2a09cc0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:15.741446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067646dd-279e-414f-8565-7883ff0701b6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:15.749356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde962b-b71f-4bc2-a8f3-9d6968f73f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:15.750050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde962b-b71f-4bc2-a8f3-9d6968f73f8b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.750809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde962b-b71f-4bc2-a8f3-9d6968f73f8b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.751091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde962b-b71f-4bc2-a8f3-9d6968f73f8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.764917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde962b-b71f-4bc2-a8f3-9d6968f73f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-0cde962b-b71f-4bc2-a8f3-9d6968f73f8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:15.773448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:15.775050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:15.775814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 86/162] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:15.776445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae6215-a625-44c9-b176-399b8fa77187 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.777249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae6215-a625-44c9-b176-399b8fa77187 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.777249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae6215-a625-44c9-b176-399b8fa77187 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.778650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 87/163] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:15.779229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae6215-a625-44c9-b176-399b8fa77187 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:15.783315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a014ccc-b0ee-46de-ba9c-3629e20b57dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:15.784272 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a014ccc-b0ee-46de-ba9c-3629e20b57dd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.784752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a014ccc-b0ee-46de-ba9c-3629e20b57dd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.785220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a014ccc-b0ee-46de-ba9c-3629e20b57dd None admin] PATH_INFO: `/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.786633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a014ccc-b0ee-46de-ba9c-3629e20b57dd None admin] RBAC: Authorizing `identity:get_user(user_id=f59916fe4d6d4a6aac1fc1ff790ac163)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:15.788286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a014ccc-b0ee-46de-ba9c-3629e20b57dd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:15.790689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 77/164] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:15 2022] GET /identity/v3/users/f59916fe4d6d4a6aac1fc1ff790ac163 => generated 248 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:15.792043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce563d1-2ab4-4565-8086-ee9036dd1791 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.792514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce563d1-2ab4-4565-8086-ee9036dd1791 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.792986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce563d1-2ab4-4565-8086-ee9036dd1791 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.794653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 78/165] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:15.799352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce563d1-2ab4-4565-8086-ee9036dd1791 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:15.846415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:15.881819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:15.916732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae6215-a625-44c9-b176-399b8fa77187 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:15.918316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce563d1-2ab4-4565-8086-ee9036dd1791 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:15.955381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d313c-346a-4590-92c5-23bb19f27ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:15.956475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d313c-346a-4590-92c5-23bb19f27ff2 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.956965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d313c-346a-4590-92c5-23bb19f27ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.957484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d313c-346a-4590-92c5-23bb19f27ff2 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1/users/3c46f5d361e8434bb7b3b04d47b8bf8c/roles/b806354bf1c840cb9329d2e65a7576f2` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.959845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d313c-346a-4590-92c5-23bb19f27ff2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3aa2c7542ee14875b5538c96394941d1, user_id=3c46f5d361e8434bb7b3b04d47b8bf8c, role_id=b806354bf1c840cb9329d2e65a7576f2)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:15.961851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d313c-346a-4590-92c5-23bb19f27ff2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:15.968459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4267c-6445-4e4b-9556-993e84c44fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:15.969159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4267c-6445-4e4b-9556-993e84c44fa6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.969468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4267c-6445-4e4b-9556-993e84c44fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.969751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4267c-6445-4e4b-9556-993e84c44fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.970507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4267c-6445-4e4b-9556-993e84c44fa6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:15.970809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4267c-6445-4e4b-9556-993e84c44fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:15.972196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4267c-6445-4e4b-9556-993e84c44fa6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:15.985535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 79/167] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:15 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:15.985535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c556f2f-8f2d-42be-a361-049763077acd None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:15.985535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c556f2f-8f2d-42be-a361-049763077acd None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:15.985535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c556f2f-8f2d-42be-a361-049763077acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:15.986480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 88/167] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:15 2022] PUT /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1/users/3c46f5d361e8434bb7b3b04d47b8bf8c/roles/b806354bf1c840cb9329d2e65a7576f2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:15.987852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d313c-346a-4590-92c5-23bb19f27ff2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:16.009451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-3c556f2f-8f2d-42be-a361-049763077acd None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:16.012440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3c556f2f-8f2d-42be-a361-049763077acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:16.178780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 80/168] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:16.180709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e0962-0188-4e0a-8120-99020b774874 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.181530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e0962-0188-4e0a-8120-99020b774874 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.182044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e0962-0188-4e0a-8120-99020b774874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.202339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d313c-346a-4590-92c5-23bb19f27ff2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:16.206495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-620e0962-0188-4e0a-8120-99020b774874 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:16.207931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-620e0962-0188-4e0a-8120-99020b774874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:16.245936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:16.247132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a820c48-aebe-43e0-a9ae-444ca327d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:16.247873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a820c48-aebe-43e0-a9ae-444ca327d825 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.248204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a820c48-aebe-43e0-a9ae-444ca327d825 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.248477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a820c48-aebe-43e0-a9ae-444ca327d825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.259315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:16.325515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 81/169] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:16.326755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a820c48-aebe-43e0-a9ae-444ca327d825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:16.327624 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e0962-0188-4e0a-8120-99020b774874 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:16.333631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a820c48-aebe-43e0-a9ae-444ca327d825 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:16.440976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e0962-0188-4e0a-8120-99020b774874 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:16.464099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 89/170] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:15 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:16.465004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a820c48-aebe-43e0-a9ae-444ca327d825 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:16.480222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d612dd-1f25-4e31-8cfa-65524c61b087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:16.481327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d612dd-1f25-4e31-8cfa-65524c61b087 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.481842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d612dd-1f25-4e31-8cfa-65524c61b087 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.482329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d612dd-1f25-4e31-8cfa-65524c61b087 None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.483841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d612dd-1f25-4e31-8cfa-65524c61b087 None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:16.485629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d612dd-1f25-4e31-8cfa-65524c61b087 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:16.487569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 82/171] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:16 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:16.488930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0de42-92ff-4ce1-9539-4f06fbc9f1e1 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.489458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0de42-92ff-4ce1-9539-4f06fbc9f1e1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.490056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0de42-92ff-4ce1-9539-4f06fbc9f1e1 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.491727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 83/172] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:16.492909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0de42-92ff-4ce1-9539-4f06fbc9f1e1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:16.507337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:16.529932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:16.573663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a820c48-aebe-43e0-a9ae-444ca327d825 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:16.610087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ccfbc-044c-4fff-969b-83ef3c0c5f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:16.610727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ccfbc-044c-4fff-969b-83ef3c0c5f91 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.611010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ccfbc-044c-4fff-969b-83ef3c0c5f91 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.611263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ccfbc-044c-4fff-969b-83ef3c0c5f91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.616260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0de42-92ff-4ce1-9539-4f06fbc9f1e1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:16.630842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ccfbc-044c-4fff-969b-83ef3c0c5f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-117ccfbc-044c-4fff-969b-83ef3c0c5f91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:16.638740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:16.639657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 90/173] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:16.640966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ccfbc-044c-4fff-969b-83ef3c0c5f91 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:16.668038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c07d-fe02-4ae4-853c-3a49203d734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:16.668755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c07d-fe02-4ae4-853c-3a49203d734c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.669035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c07d-fe02-4ae4-853c-3a49203d734c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.669305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c07d-fe02-4ae4-853c-3a49203d734c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.681012 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c07d-fe02-4ae4-853c-3a49203d734c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-c209c07d-fe02-4ae4-853c-3a49203d734c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:16.686068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:16.687014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:16.688487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 84/174] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:16.689781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5239e-ec75-4726-9a40-6686903d655f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.689781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5239e-ec75-4726-9a40-6686903d655f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.689781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5239e-ec75-4726-9a40-6686903d655f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.708716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e3a5239e-ec75-4726-9a40-6686903d655f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:16.709872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e3a5239e-ec75-4726-9a40-6686903d655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:16.766869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:16.772159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ccfbc-044c-4fff-969b-83ef3c0c5f91 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:16.784009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 85/175] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:16.792468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5239e-ec75-4726-9a40-6686903d655f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:16.810015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894c030-47a7-4d57-aae5-c354e9771ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:16.810667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894c030-47a7-4d57-aae5-c354e9771ef8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.810951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894c030-47a7-4d57-aae5-c354e9771ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.811216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894c030-47a7-4d57-aae5-c354e9771ef8 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.812404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894c030-47a7-4d57-aae5-c354e9771ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:16.813901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894c030-47a7-4d57-aae5-c354e9771ef8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:16.815774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 91/176] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:16 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:16.816765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5f098-d7e2-42fe-a643-409f2819eb45 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.816991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5f098-d7e2-42fe-a643-409f2819eb45 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.817267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5f098-d7e2-42fe-a643-409f2819eb45 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.818615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 92/177] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:16 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:16.819200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5f098-d7e2-42fe-a643-409f2819eb45 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:16.824623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:16.909797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5239e-ec75-4726-9a40-6686903d655f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:16.934069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5f098-d7e2-42fe-a643-409f2819eb45 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:16.951185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bff2b-f593-4814-8368-bba40ad21254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:16.951998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bff2b-f593-4814-8368-bba40ad21254 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.952435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bff2b-f593-4814-8368-bba40ad21254 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.952710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bff2b-f593-4814-8368-bba40ad21254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.973271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:16.973907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:16.974220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:16.974527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:16.975720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:16.976031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:16.977495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:16.986743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 93/178] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:16 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:16.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.021972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:17.034801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bff2b-f593-4814-8368-bba40ad21254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:17.036411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bff2b-f593-4814-8368-bba40ad21254 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:17.058081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:17.114558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed261f09-8608-429a-b599-db5db20c49d9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.151901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 86/179] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:16 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:17.151901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bff2b-f593-4814-8368-bba40ad21254 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.156740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4305d-4165-414e-927e-12e4d5158fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.157374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4305d-4165-414e-927e-12e4d5158fe7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.157692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4305d-4165-414e-927e-12e4d5158fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.157959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4305d-4165-414e-927e-12e4d5158fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.158714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4305d-4165-414e-927e-12e4d5158fe7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:17.159015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4305d-4165-414e-927e-12e4d5158fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:17.160393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4305d-4165-414e-927e-12e4d5158fe7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:17.169953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 94/180] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:17.171141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b0a2f-d42b-4e65-bd67-8dcd4347f1bb None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.171430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b0a2f-d42b-4e65-bd67-8dcd4347f1bb None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.171706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b0a2f-d42b-4e65-bd67-8dcd4347f1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.190229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-054b0a2f-d42b-4e65-bd67-8dcd4347f1bb None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:17.191423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-054b0a2f-d42b-4e65-bd67-8dcd4347f1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:17.269552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 95/181] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:17.269552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b0a2f-d42b-4e65-bd67-8dcd4347f1bb None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.269808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bff2b-f593-4814-8368-bba40ad21254 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.292965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:17.296553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:17.308889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7df7-159f-4ff1-97cb-b9d6ad79a06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.309704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7df7-159f-4ff1-97cb-b9d6ad79a06c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.310109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7df7-159f-4ff1-97cb-b9d6ad79a06c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.310490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7df7-159f-4ff1-97cb-b9d6ad79a06c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.311814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e7df7-159f-4ff1-97cb-b9d6ad79a06c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:17.313455 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e7df7-159f-4ff1-97cb-b9d6ad79a06c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:17.320008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 87/182] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:17 2022] GET /identity/v3/domains/default => generated 200 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:17.320008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7df7-159f-4ff1-97cb-b9d6ad79a06c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.386776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b0a2f-d42b-4e65-bd67-8dcd4347f1bb None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.424266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.424884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.425162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.425452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.426240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=3aa2c7542ee14875b5538c96394941d1, user.id=f59916fe4d6d4a6aac1fc1ff790ac163) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:17.426546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:17.427985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:17.429369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7df7-159f-4ff1-97cb-b9d6ad79a06c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.434916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 96/183] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:17 2022] GET /identity/v3/role_assignments?user.id=f59916fe4d6d4a6aac1fc1ff790ac163&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 610 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:17.435778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.481243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89918a1-d76f-4b73-9616-a87cb29acd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.482172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89918a1-d76f-4b73-9616-a87cb29acd43 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.482615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89918a1-d76f-4b73-9616-a87cb29acd43 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.482985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89918a1-d76f-4b73-9616-a87cb29acd43 None admin] PATH_INFO: `/v3/users/411770b6026d4746a34e34ea2439c6f8` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.484457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89918a1-d76f-4b73-9616-a87cb29acd43 None admin] RBAC: Authorizing `identity:get_user(user_id=411770b6026d4746a34e34ea2439c6f8)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:17.486093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89918a1-d76f-4b73-9616-a87cb29acd43 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:17.488840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 88/184] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:17 2022] GET /identity/v3/users/411770b6026d4746a34e34ea2439c6f8 => generated 276 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:17.489001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89918a1-d76f-4b73-9616-a87cb29acd43 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.547717 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:17.566445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb5085-9354-416b-a6ed-21377ee6b266 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.574581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:17.606836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5009b-841a-4292-b0c2-e94e62eab62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.607779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5009b-841a-4292-b0c2-e94e62eab62b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.608216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5009b-841a-4292-b0c2-e94e62eab62b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.608484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5009b-841a-4292-b0c2-e94e62eab62b None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.609744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5009b-841a-4292-b0c2-e94e62eab62b None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:17.611222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5009b-841a-4292-b0c2-e94e62eab62b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:17.612943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 97/185] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:17 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:17.613750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5009b-841a-4292-b0c2-e94e62eab62b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.637691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89918a1-d76f-4b73-9616-a87cb29acd43 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.684751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7ae17-7658-4ef7-837c-ee101ee0d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.685398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7ae17-7658-4ef7-837c-ee101ee0d435 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.685704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7ae17-7658-4ef7-837c-ee101ee0d435 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.685978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7ae17-7658-4ef7-837c-ee101ee0d435 None admin] PATH_INFO: `/v3/users/a9914329296c49479c2548b65c855405` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.705513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7ae17-7658-4ef7-837c-ee101ee0d435 None admin] RBAC: Authorizing `identity:get_user(user_id=a9914329296c49479c2548b65c855405)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:17.707950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7ae17-7658-4ef7-837c-ee101ee0d435 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:17.709896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 89/186] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:17 2022] GET /identity/v3/users/a9914329296c49479c2548b65c855405 => generated 284 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:17.710575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7ae17-7658-4ef7-837c-ee101ee0d435 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.755203 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5009b-841a-4292-b0c2-e94e62eab62b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.797090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46af420-86a8-485b-a457-154479e0d680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.797857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46af420-86a8-485b-a457-154479e0d680 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.798087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46af420-86a8-485b-a457-154479e0d680 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.798529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46af420-86a8-485b-a457-154479e0d680 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.809966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46af420-86a8-485b-a457-154479e0d680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e46af420-86a8-485b-a457-154479e0d680 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:17.815409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:17.816558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 98/187] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:17.823300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46af420-86a8-485b-a457-154479e0d680 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.835768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7ae17-7658-4ef7-837c-ee101ee0d435 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.838251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:17.852334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:17.874050 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78843a-0294-417a-a3b1-39ad12bb5ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.874680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78843a-0294-417a-a3b1-39ad12bb5ada None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.874963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78843a-0294-417a-a3b1-39ad12bb5ada None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.875296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78843a-0294-417a-a3b1-39ad12bb5ada None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:17.875994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78843a-0294-417a-a3b1-39ad12bb5ada None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:17.877501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78843a-0294-417a-a3b1-39ad12bb5ada None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:17.909457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 90/188] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:17 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:17.910593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78843a-0294-417a-a3b1-39ad12bb5ada None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:17.945961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46af420-86a8-485b-a457-154479e0d680 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:17.989294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a0cef-c837-45c1-afc1-832f390453d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:17.990406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a0cef-c837-45c1-afc1-832f390453d2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:17.990967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a0cef-c837-45c1-afc1-832f390453d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:17.991441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a0cef-c837-45c1-afc1-832f390453d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.032826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78843a-0294-417a-a3b1-39ad12bb5ada None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:18.085438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:18.086631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.086917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.087202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.087994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] RBAC: Adding query filter params (role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=3aa2c7542ee14875b5538c96394941d1, user.id=411770b6026d4746a34e34ea2439c6f8) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:18.088303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:18.088593 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a0cef-c837-45c1-afc1-832f390453d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:18.089891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:18.090437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a0cef-c837-45c1-afc1-832f390453d2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:18.096609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 91/189] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:17 2022] GET /identity/v3/role_assignments?user.id=411770b6026d4746a34e34ea2439c6f8&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=8f997477100248a9ace1cc28831caaac => generated 610 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:18.098375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:18.109106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:18.164570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:18.241183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 99/190] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:17 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:18.242548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a0cef-c837-45c1-afc1-832f390453d2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:18.262203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7592a4-a52d-4421-a662-5d7397579c75 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:18.306626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b245-66c7-4041-9dbd-d2e92613eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:18.307413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b245-66c7-4041-9dbd-d2e92613eca3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.307746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b245-66c7-4041-9dbd-d2e92613eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.308024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b245-66c7-4041-9dbd-d2e92613eca3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.357977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a0cef-c837-45c1-afc1-832f390453d2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:18.376093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:18.386146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80b245-66c7-4041-9dbd-d2e92613eca3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:18.387660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80b245-66c7-4041-9dbd-d2e92613eca3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:18.396018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecb3fb-b117-40b6-93ff-43c88ce0a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:18.396780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecb3fb-b117-40b6-93ff-43c88ce0a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.397159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecb3fb-b117-40b6-93ff-43c88ce0a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.397586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecb3fb-b117-40b6-93ff-43c88ce0a9a0 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.398962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecb3fb-b117-40b6-93ff-43c88ce0a9a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:18.399814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecb3fb-b117-40b6-93ff-43c88ce0a9a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:18.401089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecb3fb-b117-40b6-93ff-43c88ce0a9a0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:18.410218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 100/191] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:18 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:18.411597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1f11-df80-4985-8d18-de401d2ae21c None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.411984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1f11-df80-4985-8d18-de401d2ae21c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.412375 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1f11-df80-4985-8d18-de401d2ae21c None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.414080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 101/192] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:18.415647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f1f11-df80-4985-8d18-de401d2ae21c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:18.430255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:18.525775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 92/193] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:18 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:18.526764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b245-66c7-4041-9dbd-d2e92613eca3 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:18.536010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f1f11-df80-4985-8d18-de401d2ae21c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:18.581545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8425a-558f-4d3a-acea-f3321e1497c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:18.582477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8425a-558f-4d3a-acea-f3321e1497c8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.582765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8425a-558f-4d3a-acea-f3321e1497c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.583038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8425a-558f-4d3a-acea-f3321e1497c8 None admin] PATH_INFO: `/v3/projects/2840a8dcbce742e19ff5b08413bc8b4a` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.584858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8425a-558f-4d3a-acea-f3321e1497c8 None admin] RBAC: Authorizing `identity:get_project(project_id=2840a8dcbce742e19ff5b08413bc8b4a)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:18.587002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8425a-558f-4d3a-acea-f3321e1497c8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:18.589491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 102/194] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:18 2022] GET /identity/v3/projects/2840a8dcbce742e19ff5b08413bc8b4a => generated 313 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:18.590767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def79f1-d517-4f22-9327-7c313e73f647 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.591365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def79f1-d517-4f22-9327-7c313e73f647 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.591644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def79f1-d517-4f22-9327-7c313e73f647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.618049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8def79f1-d517-4f22-9327-7c313e73f647 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:18.619636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8def79f1-d517-4f22-9327-7c313e73f647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:18.649620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b245-66c7-4041-9dbd-d2e92613eca3 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:18.667715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:18.695948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70522963-22b9-4796-a89c-3a46ac87e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:18.696696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70522963-22b9-4796-a89c-3a46ac87e287 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.697067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70522963-22b9-4796-a89c-3a46ac87e287 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.697462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70522963-22b9-4796-a89c-3a46ac87e287 None admin] PATH_INFO: `/v3/users/a9914329296c49479c2548b65c855405` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.698827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70522963-22b9-4796-a89c-3a46ac87e287 None admin] RBAC: Authorizing `identity:get_user(user_id=a9914329296c49479c2548b65c855405)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:18.700408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70522963-22b9-4796-a89c-3a46ac87e287 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:18.702402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 93/195] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:18 2022] GET /identity/v3/users/a9914329296c49479c2548b65c855405 => generated 284 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:18.702952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70522963-22b9-4796-a89c-3a46ac87e287 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:18.705799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:18.706157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 103/196] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:18.707456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def79f1-d517-4f22-9327-7c313e73f647 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:18.838920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def79f1-d517-4f22-9327-7c313e73f647 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:18.846261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70522963-22b9-4796-a89c-3a46ac87e287 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:18.885841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277871a-60a0-48b1-adca-0279762f1f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:18.886721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277871a-60a0-48b1-adca-0279762f1f14 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.887154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277871a-60a0-48b1-adca-0279762f1f14 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.887409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:18.887738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277871a-60a0-48b1-adca-0279762f1f14 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.888225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:18.888687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:18.889050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:18.889651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277871a-60a0-48b1-adca-0279762f1f14 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:18.890052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] RBAC: Adding query filter params (role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=2840a8dcbce742e19ff5b08413bc8b4a, user.id=411770b6026d4746a34e34ea2439c6f8) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:18.890444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:18.891642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277871a-60a0-48b1-adca-0279762f1f14 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:18.892041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:18.893670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 94/197] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:18 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:18.900186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 104/198] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:18 2022] GET /identity/v3/role_assignments?user.id=411770b6026d4746a34e34ea2439c6f8&scope.project.id=2840a8dcbce742e19ff5b08413bc8b4a&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 610 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:18.900398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277871a-60a0-48b1-adca-0279762f1f14 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:18.901275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:18.942960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:18.996057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:19.022764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee44c-7684-4bdc-9a63-834b12009d4b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:19.038590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277871a-60a0-48b1-adca-0279762f1f14 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:19.066706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3221c40-05ab-4a41-b45f-c8df718ef998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:19.067368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3221c40-05ab-4a41-b45f-c8df718ef998 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:19.067651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3221c40-05ab-4a41-b45f-c8df718ef998 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:19.067934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3221c40-05ab-4a41-b45f-c8df718ef998 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:19.080167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113260f-a221-48d7-983b-25a6efca7ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:19.081183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113260f-a221-48d7-983b-25a6efca7ff0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:19.081690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113260f-a221-48d7-983b-25a6efca7ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:19.082212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113260f-a221-48d7-983b-25a6efca7ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:19.082618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3221c40-05ab-4a41-b45f-c8df718ef998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-c3221c40-05ab-4a41-b45f-c8df718ef998 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:19.089686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:19.090577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:19.091384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 105/199] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:19.092577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3221c40-05ab-4a41-b45f-c8df718ef998 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:19.201468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113260f-a221-48d7-983b-25a6efca7ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:19.203597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113260f-a221-48d7-983b-25a6efca7ff0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:19.228600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:19.236113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3221c40-05ab-4a41-b45f-c8df718ef998 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:19.272428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:19.274073 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:19.274703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:19.275291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:19.276280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=62daff6eeea54042a2879442094ef1b9, user.id=a9914329296c49479c2548b65c855405) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:19.276785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:19.277795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:19.278701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:19.285622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 106/200] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:19 2022] GET /identity/v3/role_assignments?user.id=a9914329296c49479c2548b65c855405&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:19.286746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:19.310350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 95/201] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:18 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:19.319177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113260f-a221-48d7-983b-25a6efca7ff0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:19.422781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4023-051d-4bdb-8a03-7252b54bd55f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:19.433175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113260f-a221-48d7-983b-25a6efca7ff0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:19.464092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:19.465166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:19.465741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:19.466222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:19.467189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:19.467742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:19.469541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:19.470670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15298847-72d9-4193-ab8f-58384894e673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:19.471322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15298847-72d9-4193-ab8f-58384894e673 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:19.471603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15298847-72d9-4193-ab8f-58384894e673 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:19.471872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15298847-72d9-4193-ab8f-58384894e673 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:19.473050 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15298847-72d9-4193-ab8f-58384894e673 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:19.474548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15298847-72d9-4193-ab8f-58384894e673 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:19.476273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 96/202] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:19 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:19.485741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 107/203] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:19 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:19.488997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15298847-72d9-4193-ab8f-58384894e673 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:19.491897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:19.522377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:19.563798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:19.636093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15298847-72d9-4193-ab8f-58384894e673 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:19.642036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d06b0-12b8-4d61-adcb-1b3a94ba4544 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:19.684197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1abbd-e16a-4413-b72e-8a9695b2270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:19.684858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1abbd-e16a-4413-b72e-8a9695b2270b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:19.685141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1abbd-e16a-4413-b72e-8a9695b2270b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:19.685478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1abbd-e16a-4413-b72e-8a9695b2270b None admin] PATH_INFO: `/v3/users/3c46f5d361e8434bb7b3b04d47b8bf8c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:19.689430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1abbd-e16a-4413-b72e-8a9695b2270b None admin] RBAC: Authorizing `identity:get_user(user_id=3c46f5d361e8434bb7b3b04d47b8bf8c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:19.689430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1abbd-e16a-4413-b72e-8a9695b2270b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:19.690723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 108/204] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:19 2022] GET /identity/v3/users/3c46f5d361e8434bb7b3b04d47b8bf8c => generated 290 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:19.696163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1abbd-e16a-4413-b72e-8a9695b2270b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:19.697131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:19.697792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:19.698069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:19.698343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:19.699140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] RBAC: Adding query filter params (role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=62daff6eeea54042a2879442094ef1b9, user.id=a9914329296c49479c2548b65c855405) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:19.699446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:19.700897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:19.717106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 97/205] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:19 2022] GET /identity/v3/role_assignments?user.id=a9914329296c49479c2548b65c855405&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=8f997477100248a9ace1cc28831caaac => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:19.811221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:19.821394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1abbd-e16a-4413-b72e-8a9695b2270b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:19.858794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde37c4e-d0c3-4d6a-91a4-a11cbe1287e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:19.859555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde37c4e-d0c3-4d6a-91a4-a11cbe1287e3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:19.859838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde37c4e-d0c3-4d6a-91a4-a11cbe1287e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:19.860113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde37c4e-d0c3-4d6a-91a4-a11cbe1287e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:19.935954 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde37c4e-d0c3-4d6a-91a4-a11cbe1287e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:19.937466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde37c4e-d0c3-4d6a-91a4-a11cbe1287e3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:20.037003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:20.048638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 109/206] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:19 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:20.054606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:20.177895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421f1c-0f58-4d67-80e6-5453f802a945 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:20.181173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:20.215627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ccf02-c221-4eb7-9627-ae51ef453adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:20.216836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ccf02-c221-4eb7-9627-ae51ef453adf None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:20.217391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ccf02-c221-4eb7-9627-ae51ef453adf None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:20.217803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ccf02-c221-4eb7-9627-ae51ef453adf None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:20.219113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ccf02-c221-4eb7-9627-ae51ef453adf None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:20.220725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ccf02-c221-4eb7-9627-ae51ef453adf None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:20.222499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 98/207] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:20 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:20.229331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde37c4e-d0c3-4d6a-91a4-a11cbe1287e3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:20.346921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde37c4e-d0c3-4d6a-91a4-a11cbe1287e3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:20.393507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4796017-da6c-4c55-9d27-24213c99ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:20.394090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796017-da6c-4c55-9d27-24213c99ca1e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:20.394368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796017-da6c-4c55-9d27-24213c99ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:20.394640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796017-da6c-4c55-9d27-24213c99ca1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:20.395683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4796017-da6c-4c55-9d27-24213c99ca1e None admin] RBAC: Adding query filter params (role.id=b806354bf1c840cb9329d2e65a7576f2, scope.project.id=3aa2c7542ee14875b5538c96394941d1, user.id=3c46f5d361e8434bb7b3b04d47b8bf8c) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:20.395987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4796017-da6c-4c55-9d27-24213c99ca1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:20.397435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4796017-da6c-4c55-9d27-24213c99ca1e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:20.409998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 110/208] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:20 2022] GET /identity/v3/role_assignments?user.id=3c46f5d361e8434bb7b3b04d47b8bf8c&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=b806354bf1c840cb9329d2e65a7576f2 => generated 610 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:20.720118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d8c4-b1bf-4427-9f92-054042e8ea5f None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:20.720620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d8c4-b1bf-4427-9f92-054042e8ea5f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:20.720620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d8c4-b1bf-4427-9f92-054042e8ea5f None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:20.722117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 99/209] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:20.730457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbb85-b48b-4b5d-8da5-99dc569411c1 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:20.730957 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbb85-b48b-4b5d-8da5-99dc569411c1 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:20.731115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbb85-b48b-4b5d-8da5-99dc569411c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:20.752695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9d2cbb85-b48b-4b5d-8da5-99dc569411c1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:20.753919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9d2cbb85-b48b-4b5d-8da5-99dc569411c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:20.815770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:20.841267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 111/210] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:20.865724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6d8c4-b1bf-4427-9f92-054042e8ea5f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:20.880303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b739e-079a-4cfd-9de2-3594163351ed None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:20.880616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b739e-079a-4cfd-9de2-3594163351ed None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:20.880856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b739e-079a-4cfd-9de2-3594163351ed None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:20.882575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 112/211] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:20.887843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3e5f5-8bfc-4e57-affd-e9a032ec4482 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:20.888123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3e5f5-8bfc-4e57-affd-e9a032ec4482 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:20.888398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3e5f5-8bfc-4e57-affd-e9a032ec4482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:20.922449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-b8d3e5f5-8bfc-4e57-affd-e9a032ec4482 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:20.924174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b8d3e5f5-8bfc-4e57-affd-e9a032ec4482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:21.001092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 113/212] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:21.002319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6d8c4-b1bf-4427-9f92-054042e8ea5f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:21.026012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3e5f5-8bfc-4e57-affd-e9a032ec4482 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:21.045059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126bdd1-2496-4439-8a1e-72a19eeb6fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:21.046188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126bdd1-2496-4439-8a1e-72a19eeb6fd0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:21.046525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126bdd1-2496-4439-8a1e-72a19eeb6fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:21.046807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126bdd1-2496-4439-8a1e-72a19eeb6fd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:21.061350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126bdd1-2496-4439-8a1e-72a19eeb6fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-c126bdd1-2496-4439-8a1e-72a19eeb6fd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:21.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:21.071132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:21.071944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 100/213] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:21.077610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126bdd1-2496-4439-8a1e-72a19eeb6fd0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:21.112526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:21.134137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3e5f5-8bfc-4e57-affd-e9a032ec4482 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:21.150036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:21.173978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629f0de-e0ca-4b80-b5eb-1550996e4396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:21.174653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f0de-e0ca-4b80-b5eb-1550996e4396 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:21.174933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f0de-e0ca-4b80-b5eb-1550996e4396 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:21.175265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f0de-e0ca-4b80-b5eb-1550996e4396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:21.176589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629f0de-e0ca-4b80-b5eb-1550996e4396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:21.178141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629f0de-e0ca-4b80-b5eb-1550996e4396 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:21.180580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 114/214] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:21 2022] GET /identity/v3/domains/default => generated 200 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:21.186223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629f0de-e0ca-4b80-b5eb-1550996e4396 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:21.197300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126bdd1-2496-4439-8a1e-72a19eeb6fd0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:21.234868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:21.235742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:21.236192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:21.236699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:21.237740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] RBAC: Adding query filter params (name=member) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:21.238240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:21.239881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:21.250281 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 101/215] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:21 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:21.256065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:21.293037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629f0de-e0ca-4b80-b5eb-1550996e4396 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:21.327781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14462816-5645-46a0-8b61-bed6946999ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:21.328406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14462816-5645-46a0-8b61-bed6946999ba None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:21.328685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14462816-5645-46a0-8b61-bed6946999ba None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:21.328954 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14462816-5645-46a0-8b61-bed6946999ba None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:21.329670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14462816-5645-46a0-8b61-bed6946999ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:21.331105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14462816-5645-46a0-8b61-bed6946999ba None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:21.356452 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:21.358043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 115/216] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:21 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:07:21.366330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d10221-c5f7-4447-9e90-9ac60eaa2245 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:21.406420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478c95e-f4f2-4f5f-8ff1-63afb4152b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:21.407180 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478c95e-f4f2-4f5f-8ff1-63afb4152b05 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:21.407537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478c95e-f4f2-4f5f-8ff1-63afb4152b05 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:21.407888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478c95e-f4f2-4f5f-8ff1-63afb4152b05 None admin] PATH_INFO: `/v3/users/3c073a3e328c486ebad268e2147eea13` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:21.425217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478c95e-f4f2-4f5f-8ff1-63afb4152b05 None admin] RBAC: Authorizing `identity:get_user(user_id=3c073a3e328c486ebad268e2147eea13)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:21.426756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478c95e-f4f2-4f5f-8ff1-63afb4152b05 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:21.428520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 102/217] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:21 2022] GET /identity/v3/users/3c073a3e328c486ebad268e2147eea13 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:21.435064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14462816-5645-46a0-8b61-bed6946999ba None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:21.509947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:21.548873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14462816-5645-46a0-8b61-bed6946999ba None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:21.586710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d433d-692a-4166-bac3-cd5f6edccdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:21.587349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d433d-692a-4166-bac3-cd5f6edccdf0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:21.587632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d433d-692a-4166-bac3-cd5f6edccdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:21.587905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d433d-692a-4166-bac3-cd5f6edccdf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:21.661493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d433d-692a-4166-bac3-cd5f6edccdf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:21.662971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d433d-692a-4166-bac3-cd5f6edccdf0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:21.706644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:21.715833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985e7c-0523-401a-a1df-7414496adc89 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:21.716253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985e7c-0523-401a-a1df-7414496adc89 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:21.716530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985e7c-0523-401a-a1df-7414496adc89 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:21.718097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 103/218] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:21.723748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae695-a2c5-4c2a-8da5-7173f8c7de60 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:21.724332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae695-a2c5-4c2a-8da5-7173f8c7de60 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:21.724332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae695-a2c5-4c2a-8da5-7173f8c7de60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:21.751438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e7fae695-a2c5-4c2a-8da5-7173f8c7de60 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:21.754312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e7fae695-a2c5-4c2a-8da5-7173f8c7de60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:21.796631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 116/219] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:21 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:21.810985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d433d-692a-4166-bac3-cd5f6edccdf0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:21.861222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 104/220] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:21.889997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fae695-a2c5-4c2a-8da5-7173f8c7de60 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:22.019373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:22.046051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fae695-a2c5-4c2a-8da5-7173f8c7de60 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:22.056555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d433d-692a-4166-bac3-cd5f6edccdf0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:22.101457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eabc91-f6b4-47d4-93d8-86b55d47e1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:22.102109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ade40-6ccb-4067-aa05-cb05e0b6df35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:22.102840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ade40-6ccb-4067-aa05-cb05e0b6df35 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.103115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ade40-6ccb-4067-aa05-cb05e0b6df35 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.103389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ade40-6ccb-4067-aa05-cb05e0b6df35 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.104063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eabc91-f6b4-47d4-93d8-86b55d47e1cf None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.104614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ade40-6ccb-4067-aa05-cb05e0b6df35 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:22.105114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eabc91-f6b4-47d4-93d8-86b55d47e1cf None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.105884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eabc91-f6b4-47d4-93d8-86b55d47e1cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.106130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ade40-6ccb-4067-aa05-cb05e0b6df35 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:22.108075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 117/221] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:21 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:22.114468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ade40-6ccb-4067-aa05-cb05e0b6df35 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:22.123851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eabc91-f6b4-47d4-93d8-86b55d47e1cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:22.125748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-d4eabc91-f6b4-47d4-93d8-86b55d47e1cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:22.130686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:22.131577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:22.132361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:22.136346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 105/222] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:22.142972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eabc91-f6b4-47d4-93d8-86b55d47e1cf None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:22.272211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eabc91-f6b4-47d4-93d8-86b55d47e1cf None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:22.311801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927ff02-807a-4d98-bc5b-a5bbb43853c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:22.312756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ff02-807a-4d98-bc5b-a5bbb43853c2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.313308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ff02-807a-4d98-bc5b-a5bbb43853c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.313865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ff02-807a-4d98-bc5b-a5bbb43853c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.313989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ade40-6ccb-4067-aa05-cb05e0b6df35 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:22.314903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927ff02-807a-4d98-bc5b-a5bbb43853c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:22.315399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927ff02-807a-4d98-bc5b-a5bbb43853c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:22.316992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927ff02-807a-4d98-bc5b-a5bbb43853c2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:22.326487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 106/223] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:22 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:22.333358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40373373-e1d0-455d-9d1b-784172d5d6cf None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.334077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40373373-e1d0-455d-9d1b-784172d5d6cf None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.334548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40373373-e1d0-455d-9d1b-784172d5d6cf None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.336565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 107/224] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:22 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:22.337498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40373373-e1d0-455d-9d1b-784172d5d6cf None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:22.349775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:22.355351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b961f7a-98c3-4842-baf6-555ef28983d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:22.356031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961f7a-98c3-4842-baf6-555ef28983d5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.356345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961f7a-98c3-4842-baf6-555ef28983d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.356578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961f7a-98c3-4842-baf6-555ef28983d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.357691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b961f7a-98c3-4842-baf6-555ef28983d5 None admin] RBAC: Adding query filter params (role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=62daff6eeea54042a2879442094ef1b9, user.id=3c073a3e328c486ebad268e2147eea13) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:22.358065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b961f7a-98c3-4842-baf6-555ef28983d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:22.359926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b961f7a-98c3-4842-baf6-555ef28983d5 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:22.367800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 118/225] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:22 2022] GET /identity/v3/role_assignments?user.id=3c073a3e328c486ebad268e2147eea13&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:22.369256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ef68-2a93-41e5-9391-eddae6fd5ef2 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.369811 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ef68-2a93-41e5-9391-eddae6fd5ef2 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.370398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ef68-2a93-41e5-9391-eddae6fd5ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.393786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7615ef68-2a93-41e5-9391-eddae6fd5ef2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:22.395305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7615ef68-2a93-41e5-9391-eddae6fd5ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:22.521759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40373373-e1d0-455d-9d1b-784172d5d6cf None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:22.530887 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:22.547332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 119/226] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:22.566601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c9a3-06dc-40bd-8fb7-57a8aa17259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:22.567210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c9a3-06dc-40bd-8fb7-57a8aa17259e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.567565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c9a3-06dc-40bd-8fb7-57a8aa17259e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.567857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c9a3-06dc-40bd-8fb7-57a8aa17259e None admin] PATH_INFO: `/v3/users/a9914329296c49479c2548b65c855405` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.569757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c9a3-06dc-40bd-8fb7-57a8aa17259e None admin] RBAC: Authorizing `identity:get_user(user_id=a9914329296c49479c2548b65c855405)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:22.571436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c9a3-06dc-40bd-8fb7-57a8aa17259e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:22.573123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 108/227] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:22 2022] GET /identity/v3/users/a9914329296c49479c2548b65c855405 => generated 284 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:22.579217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ef68-2a93-41e5-9391-eddae6fd5ef2 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:22.582728 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c9a3-06dc-40bd-8fb7-57a8aa17259e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:22.735246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:22.772997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ef68-2a93-41e5-9391-eddae6fd5ef2 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:22.813865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c9a3-06dc-40bd-8fb7-57a8aa17259e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:22.836350 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7f49d-4166-40e7-80ea-b48c23324e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:22.837196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7f49d-4166-40e7-80ea-b48c23324e3b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.837503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7f49d-4166-40e7-80ea-b48c23324e3b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.837777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7f49d-4166-40e7-80ea-b48c23324e3b None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.839705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7f49d-4166-40e7-80ea-b48c23324e3b None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:22.841137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7f49d-4166-40e7-80ea-b48c23324e3b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:22.843181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 120/228] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:22 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:22.867574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7f49d-4166-40e7-80ea-b48c23324e3b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:22.891242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84272f-b72e-463f-8121-8081d76489e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:22.891897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84272f-b72e-463f-8121-8081d76489e0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:22.892173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84272f-b72e-463f-8121-8081d76489e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:22.892441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84272f-b72e-463f-8121-8081d76489e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:22.915252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84272f-b72e-463f-8121-8081d76489e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-8a84272f-b72e-463f-8121-8081d76489e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:22.920696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:22.921630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:22.922437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 109/229] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:22.928142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84272f-b72e-463f-8121-8081d76489e0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:22.981500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:23.045323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7f49d-4166-40e7-80ea-b48c23324e3b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:23.125953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e208f1d-c53d-4248-9c2e-9dcea24f7b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:23.126628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e208f1d-c53d-4248-9c2e-9dcea24f7b4e None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:23.126901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e208f1d-c53d-4248-9c2e-9dcea24f7b4e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:23.127168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e208f1d-c53d-4248-9c2e-9dcea24f7b4e None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9/users/a9914329296c49479c2548b65c855405/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:23.132071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e208f1d-c53d-4248-9c2e-9dcea24f7b4e None admin] RBAC: Authorizing `identity:create_grant(project_id=62daff6eeea54042a2879442094ef1b9, user_id=a9914329296c49479c2548b65c855405, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:23.133569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e208f1d-c53d-4248-9c2e-9dcea24f7b4e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:23.139400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84272f-b72e-463f-8121-8081d76489e0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:23.165659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 121/230] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:22 2022] PUT /identity/v3/projects/62daff6eeea54042a2879442094ef1b9/users/a9914329296c49479c2548b65c855405/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:07:23.178857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:23.192292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:23.192943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:23.193223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:23.193515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:23.194259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:23.194559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:23.196418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:23.207523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 110/231] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:22 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:23.213525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e208f1d-c53d-4248-9c2e-9dcea24f7b4e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:23.344425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e208f1d-c53d-4248-9c2e-9dcea24f7b4e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:23.373874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:23.382154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347efad-55f1-4e52-b989-2ea3b70ffaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:23.383239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347efad-55f1-4e52-b989-2ea3b70ffaca None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:23.383554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347efad-55f1-4e52-b989-2ea3b70ffaca None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:23.383870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347efad-55f1-4e52-b989-2ea3b70ffaca None admin] PATH_INFO: `/v3/users/a9914329296c49479c2548b65c855405` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:23.385319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347efad-55f1-4e52-b989-2ea3b70ffaca None admin] RBAC: Authorizing `identity:get_user(user_id=a9914329296c49479c2548b65c855405)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:23.387154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347efad-55f1-4e52-b989-2ea3b70ffaca None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:23.389028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 122/232] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:23 2022] GET /identity/v3/users/a9914329296c49479c2548b65c855405 => generated 284 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:23.394565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:23.501995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7a80-5fb0-41e2-8c12-168458990db2 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:23.537169 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e05a6b-1879-4ade-b6c6-4040b1588ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:23.537894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e05a6b-1879-4ade-b6c6-4040b1588ed8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:23.538174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e05a6b-1879-4ade-b6c6-4040b1588ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:23.538447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e05a6b-1879-4ade-b6c6-4040b1588ed8 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:23.539690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e05a6b-1879-4ade-b6c6-4040b1588ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:23.541294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e05a6b-1879-4ade-b6c6-4040b1588ed8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:23.543287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 111/233] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:23 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:23.549102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347efad-55f1-4e52-b989-2ea3b70ffaca None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:23.681226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347efad-55f1-4e52-b989-2ea3b70ffaca None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:23.716973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7aa7b-9412-4fd3-89f1-0c68b110a06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:23.718018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7aa7b-9412-4fd3-89f1-0c68b110a06a None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:23.718503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7aa7b-9412-4fd3-89f1-0c68b110a06a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:23.718978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7aa7b-9412-4fd3-89f1-0c68b110a06a None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9/users/a9914329296c49479c2548b65c855405/roles/8f997477100248a9ace1cc28831caaac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:23.721436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7aa7b-9412-4fd3-89f1-0c68b110a06a None admin] RBAC: Authorizing `identity:create_grant(project_id=62daff6eeea54042a2879442094ef1b9, user_id=a9914329296c49479c2548b65c855405, role_id=8f997477100248a9ace1cc28831caaac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:23.723093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7aa7b-9412-4fd3-89f1-0c68b110a06a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:23.741322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 123/234] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:23 2022] PUT /identity/v3/projects/62daff6eeea54042a2879442094ef1b9/users/a9914329296c49479c2548b65c855405/roles/8f997477100248a9ace1cc28831caaac => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:23.990706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d301-347f-458b-ab04-0590ecc15252 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:23.991179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d301-347f-458b-ab04-0590ecc15252 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:23.991179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d301-347f-458b-ab04-0590ecc15252 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:23.993144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 112/235] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:24.002109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956e75-0f96-44fa-b20a-f1b0da7617ef None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.002957 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956e75-0f96-44fa-b20a-f1b0da7617ef None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.003444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956e75-0f96-44fa-b20a-f1b0da7617ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.025810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-14956e75-0f96-44fa-b20a-f1b0da7617ef None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:24.027351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-14956e75-0f96-44fa-b20a-f1b0da7617ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:24.051916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724ef0c-2dcf-465b-902c-a7d103b1d961 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.052217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724ef0c-2dcf-465b-902c-a7d103b1d961 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.052470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724ef0c-2dcf-465b-902c-a7d103b1d961 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.054211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 113/236] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:24.060744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dec25e-d82e-48f8-b9ad-ccece1e0c158 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.061025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dec25e-d82e-48f8-b9ad-ccece1e0c158 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.061298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dec25e-d82e-48f8-b9ad-ccece1e0c158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.092898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-35dec25e-d82e-48f8-b9ad-ccece1e0c158 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:24.095311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-35dec25e-d82e-48f8-b9ad-ccece1e0c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:24.101953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:24.138949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 124/237] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:24.177444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956e75-0f96-44fa-b20a-f1b0da7617ef None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:24.191646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:24.212334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 114/238] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:24.250385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dec25e-d82e-48f8-b9ad-ccece1e0c158 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:24.363938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956e75-0f96-44fa-b20a-f1b0da7617ef None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:24.434859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377c30-f8ad-4e66-ad72-f1266e3769dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:24.435884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377c30-f8ad-4e66-ad72-f1266e3769dd None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.436495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377c30-f8ad-4e66-ad72-f1266e3769dd None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.437022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377c30-f8ad-4e66-ad72-f1266e3769dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.438474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0377c30-f8ad-4e66-ad72-f1266e3769dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:24.440144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dec25e-d82e-48f8-b9ad-ccece1e0c158 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:24.440383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0377c30-f8ad-4e66-ad72-f1266e3769dd None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:24.442730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 125/239] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:24 2022] GET /identity/v3/domains/default => generated 200 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:24.450364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377c30-f8ad-4e66-ad72-f1266e3769dd None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:24.510275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ad317-c6ca-488d-b480-98f7d87a3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:24.511263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad317-c6ca-488d-b480-98f7d87a3fe6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.512776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad317-c6ca-488d-b480-98f7d87a3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.513270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad317-c6ca-488d-b480-98f7d87a3fe6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.538047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ad317-c6ca-488d-b480-98f7d87a3fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-3f6ad317-c6ca-488d-b480-98f7d87a3fe6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:24.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:24.544890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:24.545768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:24.555148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 115/240] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:24.576884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ad317-c6ca-488d-b480-98f7d87a3fe6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:24.581773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:24.635418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:24.667611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377c30-f8ad-4e66-ad72-f1266e3769dd None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:24.697243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ad317-c6ca-488d-b480-98f7d87a3fe6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:24.748079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:24.748714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.748995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.749267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.750022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:24.750322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:24.751726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:24.756558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c59db1-66d9-4026-95bc-45b008b71df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:24.757189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c59db1-66d9-4026-95bc-45b008b71df1 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.758123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c59db1-66d9-4026-95bc-45b008b71df1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.758404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c59db1-66d9-4026-95bc-45b008b71df1 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.759101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c59db1-66d9-4026-95bc-45b008b71df1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:24.760600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c59db1-66d9-4026-95bc-45b008b71df1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:24.775817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 116/241] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:24 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:24.782586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:24.809935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 126/242] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:24 2022] POST /identity/v3/users => generated 294 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:24.847248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10441-2d8b-4f12-a602-82c7a3ff66b6 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.847950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10441-2d8b-4f12-a602-82c7a3ff66b6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.848425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10441-2d8b-4f12-a602-82c7a3ff66b6 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.850069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 127/243] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:24.855819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7b23e-b859-4352-9d21-d352376fafcd None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:24.856210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7b23e-b859-4352-9d21-d352376fafcd None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:24.856517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7b23e-b859-4352-9d21-d352376fafcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:24.888025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-06d7b23e-b859-4352-9d21-d352376fafcd None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:24.889635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-06d7b23e-b859-4352-9d21-d352376fafcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:24.941735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeba60-ae49-410c-9cc1-6ad0eb956fd0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:24.962785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:25.006928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4efc92-b9bd-489d-8377-7611ff8891b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.009151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4efc92-b9bd-489d-8377-7611ff8891b1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.009658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4efc92-b9bd-489d-8377-7611ff8891b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.010135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4efc92-b9bd-489d-8377-7611ff8891b1 None admin] PATH_INFO: `/v3/users/c3a0b3edd4e04a22866c92ab638533e3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.024016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 128/244] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:25.026980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4efc92-b9bd-489d-8377-7611ff8891b1 None admin] RBAC: Authorizing `identity:get_user(user_id=c3a0b3edd4e04a22866c92ab638533e3)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:25.029391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4efc92-b9bd-489d-8377-7611ff8891b1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:25.031662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 117/245] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:24 2022] GET /identity/v3/users/c3a0b3edd4e04a22866c92ab638533e3 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:25.037288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7b23e-b859-4352-9d21-d352376fafcd None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:25.049854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4efc92-b9bd-489d-8377-7611ff8891b1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:25.091605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:25.156958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7b23e-b859-4352-9d21-d352376fafcd None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:25.161432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4efc92-b9bd-489d-8377-7611ff8891b1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:25.200237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b91623-2501-469d-9145-1a0c29183139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.200607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c0dba-30b8-4494-b849-db8c6387fa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.200875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b91623-2501-469d-9145-1a0c29183139 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.201158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b91623-2501-469d-9145-1a0c29183139 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.201254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c0dba-30b8-4494-b849-db8c6387fa6e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.201461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b91623-2501-469d-9145-1a0c29183139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.201567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c0dba-30b8-4494-b849-db8c6387fa6e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.201829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c0dba-30b8-4494-b849-db8c6387fa6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.213346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c0dba-30b8-4494-b849-db8c6387fa6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-085c0dba-30b8-4494-b849-db8c6387fa6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:25.218672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:25.219630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:25.220403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 118/246] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:25.225504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c0dba-30b8-4494-b849-db8c6387fa6e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:25.261961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:25.305738 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b91623-2501-469d-9145-1a0c29183139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:25.307270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b91623-2501-469d-9145-1a0c29183139 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:25.339913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c0dba-30b8-4494-b849-db8c6387fa6e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:25.365192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:25.377876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.378520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.378807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.379084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.379809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:25.380112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:25.381541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:25.391326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 119/247] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:25 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:25.397485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:25.441657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 129/248] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 22:07:25.451432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b91623-2501-469d-9145-1a0c29183139 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:25.508776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:25.518720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ada09-fa01-4fa7-8b58-15536f7474b6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:25.556549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcf41d-bfc9-4b65-b564-9ce4b794e834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.557132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf41d-bfc9-4b65-b564-9ce4b794e834 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.557429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf41d-bfc9-4b65-b564-9ce4b794e834 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.557716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf41d-bfc9-4b65-b564-9ce4b794e834 None admin] PATH_INFO: `/v3/users/3c073a3e328c486ebad268e2147eea13` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.558937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf41d-bfc9-4b65-b564-9ce4b794e834 None admin] RBAC: Authorizing `identity:get_user(user_id=3c073a3e328c486ebad268e2147eea13)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:25.560660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf41d-bfc9-4b65-b564-9ce4b794e834 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:25.562493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 120/249] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:25 2022] GET /identity/v3/users/3c073a3e328c486ebad268e2147eea13 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:25.567959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcf41d-bfc9-4b65-b564-9ce4b794e834 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:25.571601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b91623-2501-469d-9145-1a0c29183139 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:25.611306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553ea4-d6b0-472f-a5b0-054d088ed225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.612351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553ea4-d6b0-472f-a5b0-054d088ed225 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.612863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553ea4-d6b0-472f-a5b0-054d088ed225 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.613359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553ea4-d6b0-472f-a5b0-054d088ed225 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.615747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83553ea4-d6b0-472f-a5b0-054d088ed225 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:25.618180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83553ea4-d6b0-472f-a5b0-054d088ed225 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:25.620610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 130/250] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:25 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:25.626669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553ea4-d6b0-472f-a5b0-054d088ed225 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:25.639962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:25.685333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcf41d-bfc9-4b65-b564-9ce4b794e834 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:25.722373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47b969-8fd7-4b0b-bc71-572d333e5a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.723269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b969-8fd7-4b0b-bc71-572d333e5a32 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.723698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b969-8fd7-4b0b-bc71-572d333e5a32 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.724146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b969-8fd7-4b0b-bc71-572d333e5a32 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.725607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47b969-8fd7-4b0b-bc71-572d333e5a32 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:25.728033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47b969-8fd7-4b0b-bc71-572d333e5a32 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:25.730226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 121/251] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:25 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:25.736457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47b969-8fd7-4b0b-bc71-572d333e5a32 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:25.767064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:25.787297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553ea4-d6b0-472f-a5b0-054d088ed225 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:25.847305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47b969-8fd7-4b0b-bc71-572d333e5a32 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:25.874150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d6080-1c0b-41da-8afc-8ab478f2d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.874773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d6080-1c0b-41da-8afc-8ab478f2d47d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.875053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d6080-1c0b-41da-8afc-8ab478f2d47d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.875319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d6080-1c0b-41da-8afc-8ab478f2d47d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.876120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d6080-1c0b-41da-8afc-8ab478f2d47d None admin] RBAC: Adding query filter params (role.id=b806354bf1c840cb9329d2e65a7576f2, scope.project.id=62daff6eeea54042a2879442094ef1b9, user.id=c3a0b3edd4e04a22866c92ab638533e3) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:25.876423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d6080-1c0b-41da-8afc-8ab478f2d47d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:25.877893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d6080-1c0b-41da-8afc-8ab478f2d47d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:25.885990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af274a9-3af6-424f-bc74-5a34b8f08b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:25.886936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af274a9-3af6-424f-bc74-5a34b8f08b4e None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.887396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af274a9-3af6-424f-bc74-5a34b8f08b4e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.887846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af274a9-3af6-424f-bc74-5a34b8f08b4e None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9/users/3c073a3e328c486ebad268e2147eea13/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.890051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af274a9-3af6-424f-bc74-5a34b8f08b4e None admin] RBAC: Authorizing `identity:create_grant(project_id=62daff6eeea54042a2879442094ef1b9, user_id=3c073a3e328c486ebad268e2147eea13, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:25.891270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 131/252] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:25 2022] GET /identity/v3/role_assignments?user.id=c3a0b3edd4e04a22866c92ab638533e3&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=b806354bf1c840cb9329d2e65a7576f2 => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:25.892195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af274a9-3af6-424f-bc74-5a34b8f08b4e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:25.892519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b651-ceec-4aae-b188-bd1fc3695be0 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.892806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b651-ceec-4aae-b188-bd1fc3695be0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.893080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b651-ceec-4aae-b188-bd1fc3695be0 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.894337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 132/253] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:25.901377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b12d-a326-463d-b538-cb8f5cd09768 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:25.901894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b12d-a326-463d-b538-cb8f5cd09768 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:25.902126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b12d-a326-463d-b538-cb8f5cd09768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:25.910273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 122/254] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:25 2022] PUT /identity/v3/projects/62daff6eeea54042a2879442094ef1b9/users/3c073a3e328c486ebad268e2147eea13/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:07:25.946887 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9f14b12d-a326-463d-b538-cb8f5cd09768 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:25.948144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9f14b12d-a326-463d-b538-cb8f5cd09768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:26.053976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 133/255] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:26.089129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af274a9-3af6-424f-bc74-5a34b8f08b4e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:26.197367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:26.212893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af274a9-3af6-424f-bc74-5a34b8f08b4e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:26.254654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66372931-4153-43b2-8ab2-11d7b15d5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:26.255680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66372931-4153-43b2-8ab2-11d7b15d5736 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:26.256155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66372931-4153-43b2-8ab2-11d7b15d5736 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:26.256622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66372931-4153-43b2-8ab2-11d7b15d5736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:26.269217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66372931-4153-43b2-8ab2-11d7b15d5736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-66372931-4153-43b2-8ab2-11d7b15d5736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:26.275392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:26.276265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:26.277035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:26.279360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 123/256] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:26.285274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14b12d-a326-463d-b538-cb8f5cd09768 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:26.289764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:26.394273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14b12d-a326-463d-b538-cb8f5cd09768 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:26.421461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed9b0-7fc6-4171-afdf-11b378e9b6f8 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:26.421776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed9b0-7fc6-4171-afdf-11b378e9b6f8 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:26.422004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed9b0-7fc6-4171-afdf-11b378e9b6f8 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:26.423808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 124/257] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:26.431677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da2ab-6db3-4d2e-bb53-7297d1bde472 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:26.432206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da2ab-6db3-4d2e-bb53-7297d1bde472 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:26.432585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da2ab-6db3-4d2e-bb53-7297d1bde472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:26.436226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:26.437016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:26.437486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:26.437905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:26.438765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:26.445514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:26.447381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:26.452486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6b6da2ab-6db3-4d2e-bb53-7297d1bde472 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:26.454349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6b6da2ab-6db3-4d2e-bb53-7297d1bde472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:26.457993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 134/258] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:26 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:26.474894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:26.536534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 125/259] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:26.614139 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6da2ab-6db3-4d2e-bb53-7297d1bde472 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:26.625908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:26.672619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7a48c-5209-4b27-9e26-88c18f8a412a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:26.721926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:26.739040 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a458e9-fc2d-4f6e-957f-650423f390e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:26.740020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a458e9-fc2d-4f6e-957f-650423f390e4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:26.740527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a458e9-fc2d-4f6e-957f-650423f390e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:26.741020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a458e9-fc2d-4f6e-957f-650423f390e4 None admin] PATH_INFO: `/v3/users/a9914329296c49479c2548b65c855405` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:26.743109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a458e9-fc2d-4f6e-957f-650423f390e4 None admin] RBAC: Authorizing `identity:get_user(user_id=a9914329296c49479c2548b65c855405)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:26.744758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a458e9-fc2d-4f6e-957f-650423f390e4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:26.745964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6da2ab-6db3-4d2e-bb53-7297d1bde472 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:26.746886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 135/260] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:26 2022] GET /identity/v3/users/a9914329296c49479c2548b65c855405 => generated 284 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:26.757110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a458e9-fc2d-4f6e-957f-650423f390e4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:26.797255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b788cc-c1e8-471a-85a4-215850204544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:26.798272 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b788cc-c1e8-471a-85a4-215850204544 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:26.798621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b788cc-c1e8-471a-85a4-215850204544 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:26.798917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b788cc-c1e8-471a-85a4-215850204544 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:26.811880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b788cc-c1e8-471a-85a4-215850204544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-30b788cc-c1e8-471a-85a4-215850204544 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:26.817871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:26.819612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 126/261] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:26.825831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b788cc-c1e8-471a-85a4-215850204544 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:26.942990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b788cc-c1e8-471a-85a4-215850204544 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:26.982527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:26.983311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:26.983598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:26.983876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:26.984628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:26.984934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:26.986651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:26.993943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:26.998837 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 127/262] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:26 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:27.004545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:27.072749 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a458e9-fc2d-4f6e-957f-650423f390e4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:27.132599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88284f6c-e5bf-4054-9300-06a47906e3c9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:27.173070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6996a6-ebdc-4502-857b-1196d0adc3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:27.174142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6996a6-ebdc-4502-857b-1196d0adc3c0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:27.174661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6996a6-ebdc-4502-857b-1196d0adc3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:27.175134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6996a6-ebdc-4502-857b-1196d0adc3c0 None admin] PATH_INFO: `/v3/users/a9914329296c49479c2548b65c855405` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:27.176553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6996a6-ebdc-4502-857b-1196d0adc3c0 None admin] RBAC: Authorizing `identity:get_user(user_id=a9914329296c49479c2548b65c855405)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:27.178384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6996a6-ebdc-4502-857b-1196d0adc3c0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:27.181155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 128/263] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:27 2022] GET /identity/v3/users/a9914329296c49479c2548b65c855405 => generated 284 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:27.185303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:27.188498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6996a6-ebdc-4502-857b-1196d0adc3c0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:27.210811 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d4628-1181-4f47-b7ac-104a76fca6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:27.211465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d4628-1181-4f47-b7ac-104a76fca6b5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:27.211810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d4628-1181-4f47-b7ac-104a76fca6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:27.212155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d4628-1181-4f47-b7ac-104a76fca6b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:27.281793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:27.302969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6996a6-ebdc-4502-857b-1196d0adc3c0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:27.345945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42270b9c-b8c9-4542-9e8f-4cbb4f78b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:27.347256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42270b9c-b8c9-4542-9e8f-4cbb4f78b67a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:27.347745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42270b9c-b8c9-4542-9e8f-4cbb4f78b67a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:27.348210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42270b9c-b8c9-4542-9e8f-4cbb4f78b67a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:27.425221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42270b9c-b8c9-4542-9e8f-4cbb4f78b67a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:27.427154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42270b9c-b8c9-4542-9e8f-4cbb4f78b67a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:27.446303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d4628-1181-4f47-b7ac-104a76fca6b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:27.447795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d4628-1181-4f47-b7ac-104a76fca6b5 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:27.501964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:27.541078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 129/264] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:27 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 22:07:27.547142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42270b9c-b8c9-4542-9e8f-4cbb4f78b67a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:27.670701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42270b9c-b8c9-4542-9e8f-4cbb4f78b67a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:27.709990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85bbc3-7e44-4be6-b59c-92f38ebe27cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:27.712000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85bbc3-7e44-4be6-b59c-92f38ebe27cc None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:27.712500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85bbc3-7e44-4be6-b59c-92f38ebe27cc None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:27.712971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85bbc3-7e44-4be6-b59c-92f38ebe27cc None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:27.714445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85bbc3-7e44-4be6-b59c-92f38ebe27cc None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:27.716234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85bbc3-7e44-4be6-b59c-92f38ebe27cc None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:27.718738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 130/265] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:27 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:27.728232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85bbc3-7e44-4be6-b59c-92f38ebe27cc None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:27.774114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:27.811741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 136/266] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:26 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 1055 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:27.818384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d4628-1181-4f47-b7ac-104a76fca6b5 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:27.840610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85bbc3-7e44-4be6-b59c-92f38ebe27cc None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:27.868473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:27.880822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:27.881973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:27.883238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:27.883847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:27.885492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] RBAC: Adding query filter params (role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=62daff6eeea54042a2879442094ef1b9, user.id=a9914329296c49479c2548b65c855405) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:27.886006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:27.887668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:27.894994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 131/267] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:27 2022] GET /identity/v3/role_assignments?user.id=a9914329296c49479c2548b65c855405&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=8f997477100248a9ace1cc28831caaac => generated 610 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:27.982392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d4628-1181-4f47-b7ac-104a76fca6b5 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:28.034034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7baaf6-a52f-41c0-86d8-dee66860cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:28.034618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7baaf6-a52f-41c0-86d8-dee66860cd62 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:28.034902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7baaf6-a52f-41c0-86d8-dee66860cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:28.035178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7baaf6-a52f-41c0-86d8-dee66860cd62 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:28.043327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7baaf6-a52f-41c0-86d8-dee66860cd62 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:28.043849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7baaf6-a52f-41c0-86d8-dee66860cd62 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:28.046007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 137/268] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:27 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:28.051461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:28.219215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b15bff-f110-4a42-8bbc-e0451e1177e2 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:28.221636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:28.264010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27836f28-e0a3-4bf9-ad7d-34f3ea36cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:28.264639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27836f28-e0a3-4bf9-ad7d-34f3ea36cd58 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:28.264921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27836f28-e0a3-4bf9-ad7d-34f3ea36cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:28.265194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27836f28-e0a3-4bf9-ad7d-34f3ea36cd58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:28.266013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27836f28-e0a3-4bf9-ad7d-34f3ea36cd58 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=62daff6eeea54042a2879442094ef1b9, user.id=a9914329296c49479c2548b65c855405) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:28.266323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27836f28-e0a3-4bf9-ad7d-34f3ea36cd58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:28.267784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27836f28-e0a3-4bf9-ad7d-34f3ea36cd58 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:28.275317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 132/269] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:28 2022] GET /identity/v3/role_assignments?user.id=a9914329296c49479c2548b65c855405&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 610 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:28.354143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026f883-0128-4674-9aa9-2ce6e687237c None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:28.354663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026f883-0128-4674-9aa9-2ce6e687237c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:28.354663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026f883-0128-4674-9aa9-2ce6e687237c None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:28.356370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 138/270] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:28.362836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd61cd3-bbe8-47fd-92f0-9cb5f652097d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:28.363140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd61cd3-bbe8-47fd-92f0-9cb5f652097d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:28.363411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd61cd3-bbe8-47fd-92f0-9cb5f652097d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:28.383638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-5fd61cd3-bbe8-47fd-92f0-9cb5f652097d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:28.384848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5fd61cd3-bbe8-47fd-92f0-9cb5f652097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:28.459974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 133/271] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:28.499368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026f883-0128-4674-9aa9-2ce6e687237c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:28.630872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:28.639747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026f883-0128-4674-9aa9-2ce6e687237c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:28.683440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3340b-540f-46d4-9f5a-2b4bfa282a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:28.684110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3340b-540f-46d4-9f5a-2b4bfa282a73 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:28.684439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3340b-540f-46d4-9f5a-2b4bfa282a73 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:28.684726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3340b-540f-46d4-9f5a-2b4bfa282a73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:28.685967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3340b-540f-46d4-9f5a-2b4bfa282a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:28.687403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3340b-540f-46d4-9f5a-2b4bfa282a73 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:28.689321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 139/272] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:28 2022] GET /identity/v3/domains/default => generated 200 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:28.696607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd61cd3-bbe8-47fd-92f0-9cb5f652097d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:28.711389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef6094-2408-4604-8430-4b8ccada5672 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:28.711848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef6094-2408-4604-8430-4b8ccada5672 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:28.712130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef6094-2408-4604-8430-4b8ccada5672 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:28.713730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 140/273] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:28 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:28.720785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3a45-8e94-45b5-a7fe-4e82b8e92b2a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:28.721427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3a45-8e94-45b5-a7fe-4e82b8e92b2a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:28.721897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3a45-8e94-45b5-a7fe-4e82b8e92b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:28.744726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-09ad3a45-8e94-45b5-a7fe-4e82b8e92b2a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:28.746122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-09ad3a45-8e94-45b5-a7fe-4e82b8e92b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:28.825592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:28.834388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd61cd3-bbe8-47fd-92f0-9cb5f652097d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:28.849438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 141/274] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:28.870444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad3a45-8e94-45b5-a7fe-4e82b8e92b2a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:28.881921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd5a7-beb2-4758-b53c-f104e250b0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:28.882210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd5a7-beb2-4758-b53c-f104e250b0de None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:28.882892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd5a7-beb2-4758-b53c-f104e250b0de None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:28.882892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd5a7-beb2-4758-b53c-f104e250b0de None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:28.883519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cd5a7-beb2-4758-b53c-f104e250b0de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:28.885008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cd5a7-beb2-4758-b53c-f104e250b0de None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:28.909577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 134/275] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:28 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:28.989285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:28.995921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad3a45-8e94-45b5-a7fe-4e82b8e92b2a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.012899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673fc63-f8ce-4dca-9c1c-e4fc8bd5cf0c None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.013235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673fc63-f8ce-4dca-9c1c-e4fc8bd5cf0c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.013549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673fc63-f8ce-4dca-9c1c-e4fc8bd5cf0c None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.015246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 135/276] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:29.021010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d740b-80e7-4942-b490-f0c675573abe None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.021281 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d740b-80e7-4942-b490-f0c675573abe None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.021622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d740b-80e7-4942-b490-f0c675573abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.040052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43461ef5-975b-479b-8a2b-699420ed42cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:29.041160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461ef5-975b-479b-8a2b-699420ed42cb None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.041672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461ef5-975b-479b-8a2b-699420ed42cb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.042143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461ef5-975b-479b-8a2b-699420ed42cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.043599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f47d740b-80e7-4942-b490-f0c675573abe None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:29.046166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f47d740b-80e7-4942-b490-f0c675573abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:29.058368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43461ef5-975b-479b-8a2b-699420ed42cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-43461ef5-975b-479b-8a2b-699420ed42cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:29.063877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:29.064865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:29.065691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 142/277] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:29.073007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43461ef5-975b-479b-8a2b-699420ed42cb None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:29.153520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 136/278] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:29.154813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d85e-6d26-4c9f-9db0-164d3ca13bd7 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.155279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d85e-6d26-4c9f-9db0-164d3ca13bd7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.155279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d85e-6d26-4c9f-9db0-164d3ca13bd7 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.158257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 137/279] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:29 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:29.163332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69be14b-d8b6-4332-9afa-ec224d86ff65 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.164627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69be14b-d8b6-4332-9afa-ec224d86ff65 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.164627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69be14b-d8b6-4332-9afa-ec224d86ff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.184698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f69be14b-d8b6-4332-9afa-ec224d86ff65 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:29.185955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f69be14b-d8b6-4332-9afa-ec224d86ff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:29.210738 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43461ef5-975b-479b-8a2b-699420ed42cb None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.265196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:29.265842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.266117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.266384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.267093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:29.267385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:29.268242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 138/280] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:29.269068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:29.269174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69be14b-d8b6-4332-9afa-ec224d86ff65 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:29.286339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 143/281] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:29 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:29.287826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:29.293864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:29.321063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:29.400477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69be14b-d8b6-4332-9afa-ec224d86ff65 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.438442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0cdf7-a908-4e70-8d32-c90c896a68d9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.440300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d2fa8-fc63-4e43-b1a4-eaff47279dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:29.441363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d2fa8-fc63-4e43-b1a4-eaff47279dad None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.442644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d2fa8-fc63-4e43-b1a4-eaff47279dad None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.442644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d2fa8-fc63-4e43-b1a4-eaff47279dad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.455221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d2fa8-fc63-4e43-b1a4-eaff47279dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-db8d2fa8-fc63-4e43-b1a4-eaff47279dad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:29.460405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:29.461470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:29.462290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 139/282] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:29.462934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d2fa8-fc63-4e43-b1a4-eaff47279dad None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:29.487055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b498b-366e-4408-8dc5-3920d765652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:29.488101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b498b-366e-4408-8dc5-3920d765652b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.488594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b498b-366e-4408-8dc5-3920d765652b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.489329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b498b-366e-4408-8dc5-3920d765652b None admin] PATH_INFO: `/v3/users/d4d5141f96fb410db9045b16655efa29` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.509391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b498b-366e-4408-8dc5-3920d765652b None admin] RBAC: Authorizing `identity:get_user(user_id=d4d5141f96fb410db9045b16655efa29)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:29.511680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b498b-366e-4408-8dc5-3920d765652b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:29.514619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 144/283] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:29 2022] GET /identity/v3/users/d4d5141f96fb410db9045b16655efa29 => generated 294 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:29.516394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b498b-366e-4408-8dc5-3920d765652b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:29.542554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:29.577338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d2fa8-fc63-4e43-b1a4-eaff47279dad None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.620391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9269e0-74c4-421e-9898-b5893afc5b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:29.620852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9269e0-74c4-421e-9898-b5893afc5b1d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.621133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9269e0-74c4-421e-9898-b5893afc5b1d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.621421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9269e0-74c4-421e-9898-b5893afc5b1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.630995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:29.633371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9269e0-74c4-421e-9898-b5893afc5b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-ab9269e0-74c4-421e-9898-b5893afc5b1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:29.639630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:29.640485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:29.641269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:29.645950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 140/284] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:29.646730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9269e0-74c4-421e-9898-b5893afc5b1d None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:29.661913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b498b-366e-4408-8dc5-3920d765652b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.708947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:29.710137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.711418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.712280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.713483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:29.713989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:29.716454 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:29.728154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 145/285] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:29 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:29.729787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:29.759424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9269e0-74c4-421e-9898-b5893afc5b1d None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.796871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:29.805439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:29.806743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:29.807355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:29.807814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:29.809377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] RBAC: Adding query filter params (role.id=bad0ac6580d5470fb6dd23a137576b9c, user.id=d4d5141f96fb410db9045b16655efa29, scope.system=all) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:29.809927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:29.812319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:29.820529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 141/286] 10.222.0.109 () {56 vars in 1313 bytes} [Thu Aug 4 22:07:29 2022] GET /identity/v3/role_assignments?user.id=d4d5141f96fb410db9045b16655efa29&scope.system=all&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 227 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:29.821614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:29.903462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a0f61-ba3e-4af6-b3ba-dbbb17ca4ff8 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.974112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c48ec6-b2bc-46da-a689-342ab740040f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:29.999932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:30.000905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:30.001388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:30.002034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:30.002983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:30.003489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:30.005075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:30.015230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 146/287] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:29 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:30.021245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:30.023948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc960353-c1c9-4aec-a2e3-6d359953e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:30.024595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc960353-c1c9-4aec-a2e3-6d359953e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:30.024873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc960353-c1c9-4aec-a2e3-6d359953e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:30.025143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc960353-c1c9-4aec-a2e3-6d359953e2f2 None admin] PATH_INFO: `/v3/users/c3a0b3edd4e04a22866c92ab638533e3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:30.026104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:30.026402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc960353-c1c9-4aec-a2e3-6d359953e2f2 None admin] RBAC: Authorizing `identity:get_user(user_id=c3a0b3edd4e04a22866c92ab638533e3)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:30.027869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc960353-c1c9-4aec-a2e3-6d359953e2f2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:30.030290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 142/288] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:29 2022] GET /identity/v3/users/c3a0b3edd4e04a22866c92ab638533e3 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:30.042599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc960353-c1c9-4aec-a2e3-6d359953e2f2 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:30.133004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba09409-d76e-4e8b-9c91-39a4927537c9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:30.168981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbfd12-e8ce-442e-9c86-0feb677fb246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:30.169866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbfd12-e8ce-442e-9c86-0feb677fb246 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:30.170245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbfd12-e8ce-442e-9c86-0feb677fb246 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:30.170620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbfd12-e8ce-442e-9c86-0feb677fb246 None admin] PATH_INFO: `/v3/users/3c073a3e328c486ebad268e2147eea13` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:30.172126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbfd12-e8ce-442e-9c86-0feb677fb246 None admin] RBAC: Authorizing `identity:get_user(user_id=3c073a3e328c486ebad268e2147eea13)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:30.173711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbfd12-e8ce-442e-9c86-0feb677fb246 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:30.175310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:30.176520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 147/289] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:30 2022] GET /identity/v3/users/3c073a3e328c486ebad268e2147eea13 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:30.183436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbfd12-e8ce-442e-9c86-0feb677fb246 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:30.253214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc960353-c1c9-4aec-a2e3-6d359953e2f2 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:30.270211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:30.308960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbfd12-e8ce-442e-9c86-0feb677fb246 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:30.325439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ce0c9-57f0-4f5b-868d-2c8f3f28ef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:30.326094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ce0c9-57f0-4f5b-868d-2c8f3f28ef1f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:30.326377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ce0c9-57f0-4f5b-868d-2c8f3f28ef1f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:30.326650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ce0c9-57f0-4f5b-868d-2c8f3f28ef1f None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:30.327832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ce0c9-57f0-4f5b-868d-2c8f3f28ef1f None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:30.329780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ce0c9-57f0-4f5b-868d-2c8f3f28ef1f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:30.331796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 143/290] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:30 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:30.342578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ce0c9-57f0-4f5b-868d-2c8f3f28ef1f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:30.381741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d058ad-7395-4981-ab77-e3b996c3a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:30.383431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d058ad-7395-4981-ab77-e3b996c3a020 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:30.383431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d058ad-7395-4981-ab77-e3b996c3a020 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:30.383709 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d058ad-7395-4981-ab77-e3b996c3a020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:30.520451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ce0c9-57f0-4f5b-868d-2c8f3f28ef1f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:30.542564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d058ad-7395-4981-ab77-e3b996c3a020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:30.544056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d058ad-7395-4981-ab77-e3b996c3a020 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:30.553773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:30.570197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f5fae-84ff-4c51-acca-acdc75a05011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:30.570671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f5fae-84ff-4c51-acca-acdc75a05011 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:30.570949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f5fae-84ff-4c51-acca-acdc75a05011 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:30.571224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f5fae-84ff-4c51-acca-acdc75a05011 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9/users/c3a0b3edd4e04a22866c92ab638533e3/roles/b806354bf1c840cb9329d2e65a7576f2` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:30.575175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f5fae-84ff-4c51-acca-acdc75a05011 None admin] RBAC: Authorizing `identity:create_grant(project_id=62daff6eeea54042a2879442094ef1b9, user_id=c3a0b3edd4e04a22866c92ab638533e3, role_id=b806354bf1c840cb9329d2e65a7576f2)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:30.576633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f5fae-84ff-4c51-acca-acdc75a05011 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:30.603260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 144/291] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:07:30 2022] PUT /identity/v3/projects/62daff6eeea54042a2879442094ef1b9/users/c3a0b3edd4e04a22866c92ab638533e3/roles/b806354bf1c840cb9329d2e65a7576f2 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:07:30.660321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:30.704260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 148/292] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:30 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:30.709937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f5fae-84ff-4c51-acca-acdc75a05011 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:30.865968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f5fae-84ff-4c51-acca-acdc75a05011 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:30.907136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcf46a-86d9-4465-9854-8841a346dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:30.907785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf46a-86d9-4465-9854-8841a346dc89 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:30.908062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf46a-86d9-4465-9854-8841a346dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:30.908334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf46a-86d9-4465-9854-8841a346dc89 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:30.909585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcf46a-86d9-4465-9854-8841a346dc89 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:30.911190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcf46a-86d9-4465-9854-8841a346dc89 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:30.913180 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 145/293] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:30 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:30.919712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d058ad-7395-4981-ab77-e3b996c3a020 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:31.045795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d058ad-7395-4981-ab77-e3b996c3a020 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:31.085115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449daf3-41d4-4c84-b395-dc065add4818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:31.086651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449daf3-41d4-4c84-b395-dc065add4818 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:31.087144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449daf3-41d4-4c84-b395-dc065add4818 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:31.087603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449daf3-41d4-4c84-b395-dc065add4818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:31.088573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449daf3-41d4-4c84-b395-dc065add4818 None admin] RBAC: Adding query filter params (role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=62daff6eeea54042a2879442094ef1b9, user.id=3c073a3e328c486ebad268e2147eea13) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:31.089069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449daf3-41d4-4c84-b395-dc065add4818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:31.090722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449daf3-41d4-4c84-b395-dc065add4818 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:31.097289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 149/294] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:30 2022] GET /identity/v3/role_assignments?user.id=3c073a3e328c486ebad268e2147eea13&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 610 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:31.497647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3d4d9-7b1f-4ff4-b2be-91d5ebcc4541 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:31.498671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3d4d9-7b1f-4ff4-b2be-91d5ebcc4541 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:31.499144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3d4d9-7b1f-4ff4-b2be-91d5ebcc4541 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:31.501022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 146/295] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:31.506664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1b43-8f3f-4470-ae5c-a7b68f361073 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:31.507247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1b43-8f3f-4470-ae5c-a7b68f361073 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:31.507725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1b43-8f3f-4470-ae5c-a7b68f361073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:31.527231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4abb1b43-8f3f-4470-ae5c-a7b68f361073 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:31.528753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4abb1b43-8f3f-4470-ae5c-a7b68f361073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:31.559428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:31.601214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11b93-2e1b-4f18-8846-d4577681c263 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:31.601896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11b93-2e1b-4f18-8846-d4577681c263 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:31.602474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11b93-2e1b-4f18-8846-d4577681c263 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:31.604196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 147/296] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:31.610532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cc00e-2ca7-4ec0-870d-3616d8ed5e66 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:31.611079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cc00e-2ca7-4ec0-870d-3616d8ed5e66 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:31.611560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cc00e-2ca7-4ec0-870d-3616d8ed5e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:31.611955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 150/297] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:31.634543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1b43-8f3f-4470-ae5c-a7b68f361073 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:31.636580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-482cc00e-2ca7-4ec0-870d-3616d8ed5e66 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:31.637815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-482cc00e-2ca7-4ec0-870d-3616d8ed5e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:31.725907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:31.734825 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 148/298] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:31.746661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1b43-8f3f-4470-ae5c-a7b68f361073 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:31.761034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cc00e-2ca7-4ec0-870d-3616d8ed5e66 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:31.787803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92117a8-d1e7-4dd3-a54d-37d76874d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:31.789400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92117a8-d1e7-4dd3-a54d-37d76874d0ca None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:31.789906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92117a8-d1e7-4dd3-a54d-37d76874d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:31.790374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92117a8-d1e7-4dd3-a54d-37d76874d0ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:31.791755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92117a8-d1e7-4dd3-a54d-37d76874d0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:31.793503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92117a8-d1e7-4dd3-a54d-37d76874d0ca None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:31.795347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 151/299] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:31 2022] GET /identity/v3/domains/default => generated 200 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:31.802301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92117a8-d1e7-4dd3-a54d-37d76874d0ca None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:31.839663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:31.887712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cc00e-2ca7-4ec0-870d-3616d8ed5e66 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:31.934065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8db36f-95fd-4e96-9d74-dc2f9a46976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:31.934956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db36f-95fd-4e96-9d74-dc2f9a46976b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:31.935380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db36f-95fd-4e96-9d74-dc2f9a46976b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:31.935802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db36f-95fd-4e96-9d74-dc2f9a46976b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:31.944355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92117a8-d1e7-4dd3-a54d-37d76874d0ca None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:31.947717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8db36f-95fd-4e96-9d74-dc2f9a46976b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-0f8db36f-95fd-4e96-9d74-dc2f9a46976b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:31.952957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:31.954060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:31.954873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:31.956714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 149/300] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:31.964371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8db36f-95fd-4e96-9d74-dc2f9a46976b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:31.986733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed0735-fbd3-4ac3-890e-ad0ba39d0655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:31.987362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed0735-fbd3-4ac3-890e-ad0ba39d0655 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:31.987688 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed0735-fbd3-4ac3-890e-ad0ba39d0655 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:31.987953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed0735-fbd3-4ac3-890e-ad0ba39d0655 None admin] PATH_INFO: `/v3/groups` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:31.988610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed0735-fbd3-4ac3-890e-ad0ba39d0655 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:31.990780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed0735-fbd3-4ac3-890e-ad0ba39d0655 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:32.022307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 152/301] 10.222.0.109 () {60 vars in 1139 bytes} [Thu Aug 4 22:07:31 2022] POST /identity/v3/groups => generated 230 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:32.031706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:32.096563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8db36f-95fd-4e96-9d74-dc2f9a46976b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:32.146075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:32.146715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:32.146995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:32.147270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:32.148094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:32.148541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:32.150183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:32.151684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755a70d-e000-4467-aef8-31d67508b34d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:32.151978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755a70d-e000-4467-aef8-31d67508b34d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:32.152260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755a70d-e000-4467-aef8-31d67508b34d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:32.153961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 153/302] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:32.159624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfbd3f-db25-4b2f-9cd6-d36a5909223f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:32.159911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfbd3f-db25-4b2f-9cd6-d36a5909223f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:32.160162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfbd3f-db25-4b2f-9cd6-d36a5909223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:32.163135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 150/303] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:31 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:32.169797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:32.181359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-66dfbd3f-db25-4b2f-9cd6-d36a5909223f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:32.182635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-66dfbd3f-db25-4b2f-9cd6-d36a5909223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:32.268283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 154/304] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:32.292953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971b40-e519-4304-b339-16af8fca1f46 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:32.301009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfbd3f-db25-4b2f-9cd6-d36a5909223f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:32.307261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:32.326717 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:32.343091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0955b3-4c26-4586-b304-a6d9ee7a2d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:32.344078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0955b3-4c26-4586-b304-a6d9ee7a2d93 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:32.344561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0955b3-4c26-4586-b304-a6d9ee7a2d93 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:32.345054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0955b3-4c26-4586-b304-a6d9ee7a2d93 None admin] PATH_INFO: `/v3/users/f4690ebf76a9492cb10dc97d86896b63` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:32.363270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0955b3-4c26-4586-b304-a6d9ee7a2d93 None admin] RBAC: Authorizing `identity:get_user(user_id=f4690ebf76a9492cb10dc97d86896b63)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:32.365757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0955b3-4c26-4586-b304-a6d9ee7a2d93 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:32.368419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 151/305] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:32 2022] GET /identity/v3/users/f4690ebf76a9492cb10dc97d86896b63 => generated 294 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:32.375436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0955b3-4c26-4586-b304-a6d9ee7a2d93 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:32.420079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfbd3f-db25-4b2f-9cd6-d36a5909223f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:32.462370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36752ba7-aacf-4a0d-ab87-541473ec7777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:32.463015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36752ba7-aacf-4a0d-ab87-541473ec7777 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:32.463295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36752ba7-aacf-4a0d-ab87-541473ec7777 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:32.463569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36752ba7-aacf-4a0d-ab87-541473ec7777 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:32.478186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36752ba7-aacf-4a0d-ab87-541473ec7777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-36752ba7-aacf-4a0d-ab87-541473ec7777 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:32.483307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:32.484157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:32.484943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:32.489535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 155/306] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:32.490800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36752ba7-aacf-4a0d-ab87-541473ec7777 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:32.513863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0955b3-4c26-4586-b304-a6d9ee7a2d93 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:32.564122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:32.565094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:32.565817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:32.566326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:32.567931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] RBAC: Adding query filter params (role.id=b806354bf1c840cb9329d2e65a7576f2, user.id=f4690ebf76a9492cb10dc97d86896b63, scope.system=all) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:32.568642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:32.572010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:32.578324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 152/307] 10.222.0.109 () {56 vars in 1313 bytes} [Thu Aug 4 22:07:32 2022] GET /identity/v3/role_assignments?user.id=f4690ebf76a9492cb10dc97d86896b63&scope.system=all&role.id=b806354bf1c840cb9329d2e65a7576f2 => generated 227 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:32.592577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:32.614901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36752ba7-aacf-4a0d-ab87-541473ec7777 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:32.655882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:32.656523 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:32.656804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:32.657079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:32.657845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:32.658148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:32.659563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:32.670341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 156/308] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:32 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:32.676252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:32.713220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:32.799859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028d666-6551-4522-a9c2-b5cb3b5e6c3a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:32.841060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6dd7e-e61f-405b-80aa-1c412fed801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:32.842010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6dd7e-e61f-405b-80aa-1c412fed801e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:32.842452 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6dd7e-e61f-405b-80aa-1c412fed801e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:32.842889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6dd7e-e61f-405b-80aa-1c412fed801e None admin] PATH_INFO: `/v3/users/d4d5141f96fb410db9045b16655efa29` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:32.844238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6dd7e-e61f-405b-80aa-1c412fed801e None admin] RBAC: Authorizing `identity:get_user(user_id=d4d5141f96fb410db9045b16655efa29)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:32.846617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6dd7e-e61f-405b-80aa-1c412fed801e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:32.849858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 153/309] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:32 2022] GET /identity/v3/users/d4d5141f96fb410db9045b16655efa29 => generated 294 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:32.857132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:32.967757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38559a0b-9673-4d5a-8e42-aef9c1f010c2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:33.004327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4f4a0-55f8-4d8f-a854-5f5ea49efb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:33.004997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4f4a0-55f8-4d8f-a854-5f5ea49efb1c None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:33.005272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4f4a0-55f8-4d8f-a854-5f5ea49efb1c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:33.005598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4f4a0-55f8-4d8f-a854-5f5ea49efb1c None admin] PATH_INFO: `/v3/system/users/d4d5141f96fb410db9045b16655efa29/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:33.007740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4f4a0-55f8-4d8f-a854-5f5ea49efb1c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d4d5141f96fb410db9045b16655efa29, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:33.009077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4f4a0-55f8-4d8f-a854-5f5ea49efb1c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:33.021498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 157/310] 10.222.0.109 () {58 vars in 1339 bytes} [Thu Aug 4 22:07:32 2022] PUT /identity/v3/system/users/d4d5141f96fb410db9045b16655efa29/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:33.036641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19511ccb-58ca-464e-9d98-0473d56d5909 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:33.036954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19511ccb-58ca-464e-9d98-0473d56d5909 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:33.037191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19511ccb-58ca-464e-9d98-0473d56d5909 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:33.040246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 154/311] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:33.045109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d12d78-a084-46a9-a368-a2eba688ef3e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:33.045432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d12d78-a084-46a9-a368-a2eba688ef3e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:33.045743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d12d78-a084-46a9-a368-a2eba688ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:33.066382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-88d12d78-a084-46a9-a368-a2eba688ef3e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:33.067584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-88d12d78-a084-46a9-a368-a2eba688ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:33.070954 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:33.169338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 158/312] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:33.192619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19511ccb-58ca-464e-9d98-0473d56d5909 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:33.304969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:33.308971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19511ccb-58ca-464e-9d98-0473d56d5909 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:33.347117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895f7fb-2d18-4287-95e0-f495502f5bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:33.347764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895f7fb-2d18-4287-95e0-f495502f5bae None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:33.348043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895f7fb-2d18-4287-95e0-f495502f5bae None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:33.348315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895f7fb-2d18-4287-95e0-f495502f5bae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:33.359761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895f7fb-2d18-4287-95e0-f495502f5bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-b895f7fb-2d18-4287-95e0-f495502f5bae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:33.368466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:33.369322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:33.370412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 155/313] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:33.377573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d12d78-a084-46a9-a368-a2eba688ef3e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:33.484864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d12d78-a084-46a9-a368-a2eba688ef3e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:33.520785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:33.521790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:33.522283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:33.522823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:33.523747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] RBAC: Adding query filter params (name=reader) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:33.524241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:33.525832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:33.532655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:33.541544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 159/314] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:33 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:33.546472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895f7fb-2d18-4287-95e0-f495502f5bae None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:33.656710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895f7fb-2d18-4287-95e0-f495502f5bae None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:33.692111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bfbf8-d241-4045-8245-ddd36f5cf8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:33.692759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bfbf8-d241-4045-8245-ddd36f5cf8aa None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:33.693042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bfbf8-d241-4045-8245-ddd36f5cf8aa None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:33.693319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bfbf8-d241-4045-8245-ddd36f5cf8aa None admin] PATH_INFO: `/v3/users/c3a0b3edd4e04a22866c92ab638533e3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:33.694701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bfbf8-d241-4045-8245-ddd36f5cf8aa None admin] RBAC: Authorizing `identity:get_user(user_id=c3a0b3edd4e04a22866c92ab638533e3)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:33.696174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bfbf8-d241-4045-8245-ddd36f5cf8aa None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:33.698017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 156/315] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:33 2022] GET /identity/v3/users/c3a0b3edd4e04a22866c92ab638533e3 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:33.703775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:33.812921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca3316-3fb9-4b49-b599-f550388cccfc None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:33.848294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bda6b2-e71c-42bb-bf91-dfdaa4deca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:33.849370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bda6b2-e71c-42bb-bf91-dfdaa4deca1f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:33.849865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bda6b2-e71c-42bb-bf91-dfdaa4deca1f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:33.850355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bda6b2-e71c-42bb-bf91-dfdaa4deca1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:33.922230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:33.924937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bda6b2-e71c-42bb-bf91-dfdaa4deca1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:33.926433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bda6b2-e71c-42bb-bf91-dfdaa4deca1f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:34.032683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 160/316] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:33 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:34.040028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bfbf8-d241-4045-8245-ddd36f5cf8aa None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:34.069142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:34.152166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bfbf8-d241-4045-8245-ddd36f5cf8aa None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:34.187818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d27aee-8bab-439a-90fa-d0027f024b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:34.188775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d27aee-8bab-439a-90fa-d0027f024b6b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:34.188882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b65bb3-5ce7-4890-b8ef-2e7bc47151f4 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:34.189157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b65bb3-5ce7-4890-b8ef-2e7bc47151f4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:34.189448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b65bb3-5ce7-4890-b8ef-2e7bc47151f4 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:34.189804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d27aee-8bab-439a-90fa-d0027f024b6b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:34.190361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d27aee-8bab-439a-90fa-d0027f024b6b None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:34.191063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 161/317] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:34.191897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d27aee-8bab-439a-90fa-d0027f024b6b None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:34.193802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d27aee-8bab-439a-90fa-d0027f024b6b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:34.196476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ee318-6c97-4550-bdfe-bc3675897435 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:34.196726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ee318-6c97-4550-bdfe-bc3675897435 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:34.197046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ee318-6c97-4550-bdfe-bc3675897435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:34.197265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 157/318] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:34 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:34.202176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d27aee-8bab-439a-90fa-d0027f024b6b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:34.217590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2e9ee318-6c97-4550-bdfe-bc3675897435 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:34.218957 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2e9ee318-6c97-4550-bdfe-bc3675897435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:34.316889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 162/319] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:34.325926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:34.334388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d27aee-8bab-439a-90fa-d0027f024b6b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:34.340880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ee318-6c97-4550-bdfe-bc3675897435 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:34.375313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:34.376236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:34.376705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:34.377255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:34.378241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] RBAC: Adding query filter params (role.id=b806354bf1c840cb9329d2e65a7576f2, scope.project.id=62daff6eeea54042a2879442094ef1b9, user.id=c3a0b3edd4e04a22866c92ab638533e3) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:34.378729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:34.379029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:34.381107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:34.389445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 158/320] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:07:34 2022] GET /identity/v3/role_assignments?user.id=c3a0b3edd4e04a22866c92ab638533e3&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=b806354bf1c840cb9329d2e65a7576f2 => generated 610 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:34.471754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ee318-6c97-4550-bdfe-bc3675897435 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:34.515713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b814a4-cb4a-493c-bec3-26fdf94714c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:34.516384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b814a4-cb4a-493c-bec3-26fdf94714c9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:34.516663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b814a4-cb4a-493c-bec3-26fdf94714c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:34.516934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b814a4-cb4a-493c-bec3-26fdf94714c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:34.518142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b814a4-cb4a-493c-bec3-26fdf94714c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:34.519846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b814a4-cb4a-493c-bec3-26fdf94714c9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:34.521810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 163/321] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:34 2022] GET /identity/v3/domains/default => generated 200 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:34.527691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:34.638900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e903-c24d-40e7-8101-b11a77625bf9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:34.675151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104c11-9719-497a-82c6-228a0d6d2e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:34.675778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104c11-9719-497a-82c6-228a0d6d2e14 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:34.676067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104c11-9719-497a-82c6-228a0d6d2e14 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:34.676343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104c11-9719-497a-82c6-228a0d6d2e14 None admin] PATH_INFO: `/v3/groups` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:34.677025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8104c11-9719-497a-82c6-228a0d6d2e14 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:34.678499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8104c11-9719-497a-82c6-228a0d6d2e14 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:34.696499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 159/322] 10.222.0.109 () {60 vars in 1139 bytes} [Thu Aug 4 22:07:34 2022] POST /identity/v3/groups => generated 227 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:34.854956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0185c-6ae7-4bce-ae8c-4ac1e710c54f None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:34.855488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0185c-6ae7-4bce-ae8c-4ac1e710c54f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:34.855488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0185c-6ae7-4bce-ae8c-4ac1e710c54f None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:34.856997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 164/323] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:34.862240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198fa1a-5e74-4593-8a71-85d89435d757 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:34.862527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198fa1a-5e74-4593-8a71-85d89435d757 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:34.862798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198fa1a-5e74-4593-8a71-85d89435d757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:34.882889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0198fa1a-5e74-4593-8a71-85d89435d757 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:34.884090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0198fa1a-5e74-4593-8a71-85d89435d757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:34.901545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:34.962469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 160/324] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:34.985098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0185c-6ae7-4bce-ae8c-4ac1e710c54f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:35.104269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:35.110326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0185c-6ae7-4bce-ae8c-4ac1e710c54f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:35.148236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e8e9e-fef0-4ee7-967a-d7196a004ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:35.148894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e8e9e-fef0-4ee7-967a-d7196a004ee9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:35.149172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e8e9e-fef0-4ee7-967a-d7196a004ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:35.149468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e8e9e-fef0-4ee7-967a-d7196a004ee9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:35.160674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e8e9e-fef0-4ee7-967a-d7196a004ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-374e8e9e-fef0-4ee7-967a-d7196a004ee9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:35.165983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:35.166847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:35.168049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 165/325] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:35.174840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198fa1a-5e74-4593-8a71-85d89435d757 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:35.295852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198fa1a-5e74-4593-8a71-85d89435d757 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:35.335829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7099a05-f30c-4fea-99c3-ab84aba5d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:35.336811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7099a05-f30c-4fea-99c3-ab84aba5d2c5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:35.337341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7099a05-f30c-4fea-99c3-ab84aba5d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:35.337936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7099a05-f30c-4fea-99c3-ab84aba5d2c5 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:35.338903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7099a05-f30c-4fea-99c3-ab84aba5d2c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:35.339402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7099a05-f30c-4fea-99c3-ab84aba5d2c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:35.341149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7099a05-f30c-4fea-99c3-ab84aba5d2c5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:35.354965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 161/326] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:35 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:35.360278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e8e9e-fef0-4ee7-967a-d7196a004ee9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:35.362090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673d5e-e909-41a3-9d64-b0c9a55f9d5a None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:35.362318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673d5e-e909-41a3-9d64-b0c9a55f9d5a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:35.362592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673d5e-e909-41a3-9d64-b0c9a55f9d5a None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:35.364261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 162/327] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:35.373146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533765b-e02d-4d24-9091-c0a50650eee7 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:35.373146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533765b-e02d-4d24-9091-c0a50650eee7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:35.373146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533765b-e02d-4d24-9091-c0a50650eee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:35.393615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-5533765b-e02d-4d24-9091-c0a50650eee7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:35.395250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5533765b-e02d-4d24-9091-c0a50650eee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:35.423844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:35.485501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 163/328] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:35.495678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e8e9e-fef0-4ee7-967a-d7196a004ee9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:35.503697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533765b-e02d-4d24-9091-c0a50650eee7 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:35.538857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352ce79-fc32-4c4a-ab11-abeff29ab7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:35.539684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352ce79-fc32-4c4a-ab11-abeff29ab7c4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:35.539964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352ce79-fc32-4c4a-ab11-abeff29ab7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:35.540236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352ce79-fc32-4c4a-ab11-abeff29ab7c4 None admin] PATH_INFO: `/v3/users/f4690ebf76a9492cb10dc97d86896b63` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:35.541488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352ce79-fc32-4c4a-ab11-abeff29ab7c4 None admin] RBAC: Authorizing `identity:get_user(user_id=f4690ebf76a9492cb10dc97d86896b63)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:35.542973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352ce79-fc32-4c4a-ab11-abeff29ab7c4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:35.544614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 166/329] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:35 2022] GET /identity/v3/users/f4690ebf76a9492cb10dc97d86896b63 => generated 294 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:35.550371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352ce79-fc32-4c4a-ab11-abeff29ab7c4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:35.580442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:35.620028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533765b-e02d-4d24-9091-c0a50650eee7 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:35.658697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f32c-e404-4a29-808e-b8a683d75d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:35.659455 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f32c-e404-4a29-808e-b8a683d75d5f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:35.659641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f32c-e404-4a29-808e-b8a683d75d5f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:35.659909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f32c-e404-4a29-808e-b8a683d75d5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:35.666761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352ce79-fc32-4c4a-ab11-abeff29ab7c4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:35.671704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55f32c-e404-4a29-808e-b8a683d75d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-4d55f32c-e404-4a29-808e-b8a683d75d5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:35.678141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:35.679310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:35.680082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:35.684265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 164/330] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:35.685702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f32c-e404-4a29-808e-b8a683d75d5f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:35.697776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:35.704820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34d8da-9665-4be2-8aac-ac98909ff548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:35.705503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d8da-9665-4be2-8aac-ac98909ff548 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:35.705787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d8da-9665-4be2-8aac-ac98909ff548 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:35.706076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d8da-9665-4be2-8aac-ac98909ff548 None admin] PATH_INFO: `/v3/system/users/f4690ebf76a9492cb10dc97d86896b63/roles/b806354bf1c840cb9329d2e65a7576f2` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:35.707546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34d8da-9665-4be2-8aac-ac98909ff548 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f4690ebf76a9492cb10dc97d86896b63, role_id=b806354bf1c840cb9329d2e65a7576f2)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:35.708940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34d8da-9665-4be2-8aac-ac98909ff548 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:35.721182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 167/331] 10.222.0.109 () {58 vars in 1339 bytes} [Thu Aug 4 22:07:35 2022] PUT /identity/v3/system/users/f4690ebf76a9492cb10dc97d86896b63/roles/b806354bf1c840cb9329d2e65a7576f2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:35.812086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f32c-e404-4a29-808e-b8a683d75d5f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:35.852268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:35.852948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:35.853246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:35.853548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:35.854289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] RBAC: Adding query filter params (name=member) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:35.854597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:35.856027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:35.869111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 165/332] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:35 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:35.872339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34d8da-9665-4be2-8aac-ac98909ff548 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:35.977926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:35.988572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34d8da-9665-4be2-8aac-ac98909ff548 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:36.029955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68034b6f-3227-47f0-a8c8-677e8225e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:36.030621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034b6f-3227-47f0-a8c8-677e8225e558 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:36.030902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034b6f-3227-47f0-a8c8-677e8225e558 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:36.031177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034b6f-3227-47f0-a8c8-677e8225e558 None admin] PATH_INFO: `/v3/users/d4d5141f96fb410db9045b16655efa29` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:36.032393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68034b6f-3227-47f0-a8c8-677e8225e558 None admin] RBAC: Authorizing `identity:get_user(user_id=d4d5141f96fb410db9045b16655efa29)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:36.033878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68034b6f-3227-47f0-a8c8-677e8225e558 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:36.035665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 168/333] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:35 2022] GET /identity/v3/users/d4d5141f96fb410db9045b16655efa29 => generated 294 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:36.041738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:36.137159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:36.175888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d258c1f-99da-4764-97c9-aed10cbf1468 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:36.212535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120bb2f-4736-46f4-acbc-c5020f044750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:36.213164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120bb2f-4736-46f4-acbc-c5020f044750 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:36.213476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120bb2f-4736-46f4-acbc-c5020f044750 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:36.213759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120bb2f-4736-46f4-acbc-c5020f044750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:36.214556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120bb2f-4736-46f4-acbc-c5020f044750 None admin] RBAC: Adding query filter params (role.id=bad0ac6580d5470fb6dd23a137576b9c, user.id=d4d5141f96fb410db9045b16655efa29, scope.system=all) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:36.214865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120bb2f-4736-46f4-acbc-c5020f044750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:36.216323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120bb2f-4736-46f4-acbc-c5020f044750 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:36.233540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 166/334] 10.222.0.109 () {56 vars in 1313 bytes} [Thu Aug 4 22:07:36 2022] GET /identity/v3/role_assignments?user.id=d4d5141f96fb410db9045b16655efa29&scope.system=all&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 512 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:37.463524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b2f3f-8fde-404e-ad04-653cfa93112f None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:37.464005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b2f3f-8fde-404e-ad04-653cfa93112f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:37.464005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b2f3f-8fde-404e-ad04-653cfa93112f None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:37.465640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 169/335] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:37.471201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1e497-0fce-4a09-ae7a-e7257b659c10 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:37.472026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1e497-0fce-4a09-ae7a-e7257b659c10 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:37.472521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1e497-0fce-4a09-ae7a-e7257b659c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:37.499793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f8d1e497-0fce-4a09-ae7a-e7257b659c10 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:37.501360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f8d1e497-0fce-4a09-ae7a-e7257b659c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:37.591170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 167/336] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:37.638914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b2f3f-8fde-404e-ad04-653cfa93112f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:37.768590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b2f3f-8fde-404e-ad04-653cfa93112f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:37.794665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e660d-b3d6-4125-880d-117c6c0e474f None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:37.795288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e660d-b3d6-4125-880d-117c6c0e474f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:37.795789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e660d-b3d6-4125-880d-117c6c0e474f None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:37.797914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 168/337] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:37.802953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60295c-31ce-44e8-a89c-5c0e01d0270f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:37.803514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60295c-31ce-44e8-a89c-5c0e01d0270f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:37.804286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60295c-31ce-44e8-a89c-5c0e01d0270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:37.815526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb57c1-254b-485e-869e-1113bf460cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:37.816291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb57c1-254b-485e-869e-1113bf460cd5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:37.816576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb57c1-254b-485e-869e-1113bf460cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:37.816848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb57c1-254b-485e-869e-1113bf460cd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:37.834459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2d60295c-31ce-44e8-a89c-5c0e01d0270f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:37.841631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb57c1-254b-485e-869e-1113bf460cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:37.843064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2d60295c-31ce-44e8-a89c-5c0e01d0270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-5dfb57c1-254b-485e-869e-1113bf460cd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:37.846891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:37.847772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:37.848964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 170/338] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:37.877202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb57c1-254b-485e-869e-1113bf460cd5 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:37.912346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:37.917752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:37.947729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 169/339] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:37.982686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60295c-31ce-44e8-a89c-5c0e01d0270f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:38.061030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb57c1-254b-485e-869e-1113bf460cd5 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:38.115300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:38.116715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:38.117327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:38.117769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:38.118660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:38.119167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:38.125448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:38.136586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 171/340] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:37 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:38.144630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60295c-31ce-44e8-a89c-5c0e01d0270f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:38.165767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:38.203058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0170c7-878f-4686-aa03-23fe8c7e9147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:38.203722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0170c7-878f-4686-aa03-23fe8c7e9147 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:38.204002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0170c7-878f-4686-aa03-23fe8c7e9147 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:38.204279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0170c7-878f-4686-aa03-23fe8c7e9147 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:38.225296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0170c7-878f-4686-aa03-23fe8c7e9147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-4c0170c7-878f-4686-aa03-23fe8c7e9147 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:38.231015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:38.231877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:38.232709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:38.236804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 170/341] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:38.242822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0170c7-878f-4686-aa03-23fe8c7e9147 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:38.309800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:38.325354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:38.373822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23525df9-2c7b-4a71-a480-6d1fe4a6b653 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:38.374791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0170c7-878f-4686-aa03-23fe8c7e9147 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:38.414097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:38.416184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:38.416946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7841cc2-ef4c-4d80-a5c2-4ddf362ec3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:38.417383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:38.417812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:38.418560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7841cc2-ef4c-4d80-a5c2-4ddf362ec3e2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:38.419416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:38.419844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7841cc2-ef4c-4d80-a5c2-4ddf362ec3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:38.420256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7841cc2-ef4c-4d80-a5c2-4ddf362ec3e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:38.420694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:38.422807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:38.447990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 171/342] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:38 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:38.463025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:38.560344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7841cc2-ef4c-4d80-a5c2-4ddf362ec3e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:38.562931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7841cc2-ef4c-4d80-a5c2-4ddf362ec3e2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:38.645647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472803-aa13-42de-ae72-9712b63308c0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:38.648805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:38.695915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63ec07-d235-41c1-89c9-2a9a0adc8a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:38.696562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63ec07-d235-41c1-89c9-2a9a0adc8a51 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:38.696845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63ec07-d235-41c1-89c9-2a9a0adc8a51 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:38.697121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63ec07-d235-41c1-89c9-2a9a0adc8a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:38.745235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:38.804847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 172/343] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:38 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:38.815795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7841cc2-ef4c-4d80-a5c2-4ddf362ec3e2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:38.818539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63ec07-d235-41c1-89c9-2a9a0adc8a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:38.820783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63ec07-d235-41c1-89c9-2a9a0adc8a51 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:38.979934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7841cc2-ef4c-4d80-a5c2-4ddf362ec3e2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:39.020192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4df17-b52d-4823-9e16-1bc06b987534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:39.020884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4df17-b52d-4823-9e16-1bc06b987534 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.021161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4df17-b52d-4823-9e16-1bc06b987534 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.021476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4df17-b52d-4823-9e16-1bc06b987534 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.023280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4df17-b52d-4823-9e16-1bc06b987534 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:39.024734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4df17-b52d-4823-9e16-1bc06b987534 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:39.026635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 173/344] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:38 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:39.032648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4df17-b52d-4823-9e16-1bc06b987534 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:39.055016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 172/345] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:38 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 22:07:39.061992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63ec07-d235-41c1-89c9-2a9a0adc8a51 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:39.071344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:39.091906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:39.143296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4df17-b52d-4823-9e16-1bc06b987534 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:39.181187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d49a1-f593-49eb-b16b-5ab921f30917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:39.181871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d49a1-f593-49eb-b16b-5ab921f30917 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.182157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d49a1-f593-49eb-b16b-5ab921f30917 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.182432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d49a1-f593-49eb-b16b-5ab921f30917 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.188936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d49a1-f593-49eb-b16b-5ab921f30917 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:39.190425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d49a1-f593-49eb-b16b-5ab921f30917 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:39.192172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 174/346] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:39.193347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6c03-a2f0-4f49-8a9b-d004cb89066a None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.193651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6c03-a2f0-4f49-8a9b-d004cb89066a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.193926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6c03-a2f0-4f49-8a9b-d004cb89066a None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.195844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 175/347] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:39.201670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6c03-a2f0-4f49-8a9b-d004cb89066a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:39.283830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63ec07-d235-41c1-89c9-2a9a0adc8a51 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:39.314308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6c03-a2f0-4f49-8a9b-d004cb89066a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:39.322046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02556-b19e-47d6-a69b-18fe8cf40213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:39.322699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02556-b19e-47d6-a69b-18fe8cf40213 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.322976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02556-b19e-47d6-a69b-18fe8cf40213 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.323247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02556-b19e-47d6-a69b-18fe8cf40213 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.324439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc02556-b19e-47d6-a69b-18fe8cf40213 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:39.325947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc02556-b19e-47d6-a69b-18fe8cf40213 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:39.327829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 173/348] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:39.328968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89b735-4f92-48bb-8aac-5c08b815a28d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.329243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89b735-4f92-48bb-8aac-5c08b815a28d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.329767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:39.330605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89b735-4f92-48bb-8aac-5c08b815a28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.352663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-fb89b735-4f92-48bb-8aac-5c08b815a28d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:39.353719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2504c0-b64c-4931-861e-6bfc14977fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:39.353924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fb89b735-4f92-48bb-8aac-5c08b815a28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:39.354401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2504c0-b64c-4931-861e-6bfc14977fe2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.354723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2504c0-b64c-4931-861e-6bfc14977fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.355011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2504c0-b64c-4931-861e-6bfc14977fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.355815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2504c0-b64c-4931-861e-6bfc14977fe2 None admin] RBAC: Adding query filter params (group.id=897a344a4ae4412fb3c689cbce72ce41, role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=3aa2c7542ee14875b5538c96394941d1) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:39.356123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2504c0-b64c-4931-861e-6bfc14977fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:39.357624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2504c0-b64c-4931-861e-6bfc14977fe2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:39.364075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 176/349] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/role_assignments?group.id=897a344a4ae4412fb3c689cbce72ce41&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 261 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:39.365308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c726b-7dd9-42a6-9b1e-f47541e2a0b0 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.369760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c726b-7dd9-42a6-9b1e-f47541e2a0b0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.370081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c726b-7dd9-42a6-9b1e-f47541e2a0b0 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.371753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 177/350] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:39 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:39.372347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c726b-7dd9-42a6-9b1e-f47541e2a0b0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:39.443155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 174/351] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:39.445628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35f141-c6dd-40e6-b4a1-2802358edea8 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.445921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35f141-c6dd-40e6-b4a1-2802358edea8 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.446197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35f141-c6dd-40e6-b4a1-2802358edea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.454571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:39.478614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ac35f141-c6dd-40e6-b4a1-2802358edea8 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:39.479998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ac35f141-c6dd-40e6-b4a1-2802358edea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:39.566612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c726b-7dd9-42a6-9b1e-f47541e2a0b0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:39.569193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 175/352] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:39.571199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7255b7e-66e2-4af5-89bc-ca55ba88d064 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.571199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7255b7e-66e2-4af5-89bc-ca55ba88d064 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.571199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7255b7e-66e2-4af5-89bc-ca55ba88d064 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.572740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 176/353] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:39.573613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7255b7e-66e2-4af5-89bc-ca55ba88d064 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:39.625739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a727-6c47-4bc0-9fc2-cce649e94e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:39.626514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a727-6c47-4bc0-9fc2-cce649e94e23 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.626792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a727-6c47-4bc0-9fc2-cce649e94e23 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.627065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a727-6c47-4bc0-9fc2-cce649e94e23 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.628968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08a727-6c47-4bc0-9fc2-cce649e94e23 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:39.630630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08a727-6c47-4bc0-9fc2-cce649e94e23 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:39.633153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 178/354] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:39.634937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e053cc-a1ef-489e-8116-ccd0d91a6b31 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.635356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e053cc-a1ef-489e-8116-ccd0d91a6b31 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.635749 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e053cc-a1ef-489e-8116-ccd0d91a6b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.664805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-14e053cc-a1ef-489e-8116-ccd0d91a6b31 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:39.666051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-14e053cc-a1ef-489e-8116-ccd0d91a6b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:39.712554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7255b7e-66e2-4af5-89bc-ca55ba88d064 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:39.713140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:39.750821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:39.753500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 179/355] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:39.757780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd423a-527e-4e2e-93da-8d16047ed3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:39.758459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e053cc-a1ef-489e-8116-ccd0d91a6b31 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:39.758851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd423a-527e-4e2e-93da-8d16047ed3fd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.759306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd423a-527e-4e2e-93da-8d16047ed3fd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.759755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd423a-527e-4e2e-93da-8d16047ed3fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.772062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd423a-527e-4e2e-93da-8d16047ed3fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-c4bd423a-527e-4e2e-93da-8d16047ed3fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:39.778764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:39.779683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 22:07:39.780513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:39.782549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 177/356] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:39.783491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd423a-527e-4e2e-93da-8d16047ed3fd None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:39.881257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e053cc-a1ef-489e-8116-ccd0d91a6b31 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:39.906677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd423a-527e-4e2e-93da-8d16047ed3fd None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:39.940967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d24bcc-44de-4ff7-b61a-8b8de614a4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:39.941812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d24bcc-44de-4ff7-b61a-8b8de614a4c3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.942093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d24bcc-44de-4ff7-b61a-8b8de614a4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.942754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d24bcc-44de-4ff7-b61a-8b8de614a4c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.951038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bf630-9a1e-4a5b-a8c9-7475f97528bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:39.951673 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bf630-9a1e-4a5b-a8c9-7475f97528bd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.951953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bf630-9a1e-4a5b-a8c9-7475f97528bd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.952252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bf630-9a1e-4a5b-a8c9-7475f97528bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.953290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bf630-9a1e-4a5b-a8c9-7475f97528bd None admin] RBAC: Adding query filter params (group.id=897a344a4ae4412fb3c689cbce72ce41, role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=3aa2c7542ee14875b5538c96394941d1) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:39.953698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bf630-9a1e-4a5b-a8c9-7475f97528bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:39.955102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bf630-9a1e-4a5b-a8c9-7475f97528bd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:39.956295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d24bcc-44de-4ff7-b61a-8b8de614a4c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:39.961187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 178/357] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/role_assignments?group.id=897a344a4ae4412fb3c689cbce72ce41&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=8f997477100248a9ace1cc28831caaac => generated 261 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:39.962363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a368225-3e38-4ef9-a269-61bc012b4c1e None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-28d24bcc-44de-4ff7-b61a-8b8de614a4c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:39.962700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:39.963530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:39.964299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:39.964873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a368225-3e38-4ef9-a269-61bc012b4c1e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:39.964873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a368225-3e38-4ef9-a269-61bc012b4c1e None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:39.965845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 180/358] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:39.965915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 179/359] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:39.965979 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d24bcc-44de-4ff7-b61a-8b8de614a4c3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:39.966057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a368225-3e38-4ef9-a269-61bc012b4c1e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:40.007800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:40.045821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:40.097894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d24bcc-44de-4ff7-b61a-8b8de614a4c3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:40.108681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a368225-3e38-4ef9-a269-61bc012b4c1e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:40.152615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c0c4a-eccf-4920-b399-2ad8c2ee8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:40.153832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0c4a-eccf-4920-b399-2ad8c2ee8a63 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.154403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0c4a-eccf-4920-b399-2ad8c2ee8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.154958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0c4a-eccf-4920-b399-2ad8c2ee8a63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.156674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5c867-25e2-4b7c-ace6-4bcbc2c9a365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:40.157298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5c867-25e2-4b7c-ace6-4bcbc2c9a365 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.157611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5c867-25e2-4b7c-ace6-4bcbc2c9a365 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.157884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5c867-25e2-4b7c-ace6-4bcbc2c9a365 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.159050 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5c867-25e2-4b7c-ace6-4bcbc2c9a365 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:40.159227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5c867-25e2-4b7c-ace6-4bcbc2c9a365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:40.160837 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5c867-25e2-4b7c-ace6-4bcbc2c9a365 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:40.172259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c0c4a-eccf-4920-b399-2ad8c2ee8a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:40.172788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 180/360] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:40.173728 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e48e7-d55a-4962-ae3a-683c373a4e1b None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.174014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e48e7-d55a-4962-ae3a-683c373a4e1b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.174290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e48e7-d55a-4962-ae3a-683c373a4e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-6f4c0c4a-eccf-4920-b399-2ad8c2ee8a63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:40.180904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:40.181826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:40.182633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:40.185245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 181/361] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:40.186440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c0c4a-eccf-4920-b399-2ad8c2ee8a63 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:40.207944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-519e48e7-d55a-4962-ae3a-683c373a4e1b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:40.209089 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-519e48e7-d55a-4962-ae3a-683c373a4e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:40.284565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 181/362] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:40.285432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e48e7-d55a-4962-ae3a-683c373a4e1b None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:40.299445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c0c4a-eccf-4920-b399-2ad8c2ee8a63 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:40.317766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:40.338279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:40.338577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:40.339351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.339725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.340130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.340939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:40.342217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:40.343592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:40.355047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 182/363] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:40 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:40.355759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:40.396861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e48e7-d55a-4962-ae3a-683c373a4e1b None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:40.436485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dea17-a832-4d24-b6b5-7ffe7a08ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:40.437479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dea17-a832-4d24-b6b5-7ffe7a08ddbd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.437977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dea17-a832-4d24-b6b5-7ffe7a08ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.438448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dea17-a832-4d24-b6b5-7ffe7a08ddbd None admin] PATH_INFO: `/v3/users/f4690ebf76a9492cb10dc97d86896b63` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.439819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dea17-a832-4d24-b6b5-7ffe7a08ddbd None admin] RBAC: Authorizing `identity:get_user(user_id=f4690ebf76a9492cb10dc97d86896b63)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:40.441522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dea17-a832-4d24-b6b5-7ffe7a08ddbd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:40.443325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 182/364] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:07:40 2022] GET /identity/v3/users/f4690ebf76a9492cb10dc97d86896b63 => generated 294 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:40.444479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dea17-a832-4d24-b6b5-7ffe7a08ddbd None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:40.464303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9f3fc-b522-4fe0-a312-3c6e9a1548ef None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:40.508022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:40.508683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.508965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.509232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.510209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:40.510514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:40.512474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:40.523102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 183/365] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:40 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:40.525465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:40.553330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:40.565746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dea17-a832-4d24-b6b5-7ffe7a08ddbd None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:40.587086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:40.607597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d7863-b8b8-4af4-8b0a-6f939fb28427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:40.608542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d7863-b8b8-4af4-8b0a-6f939fb28427 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.609048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d7863-b8b8-4af4-8b0a-6f939fb28427 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.609555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d7863-b8b8-4af4-8b0a-6f939fb28427 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.627447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d7863-b8b8-4af4-8b0a-6f939fb28427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-c29d7863-b8b8-4af4-8b0a-6f939fb28427 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:40.635026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:40.635981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:40.636758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:40.639191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 183/366] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:40.640201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d7863-b8b8-4af4-8b0a-6f939fb28427 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:40.648705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159de3e3-86e4-4f58-b039-7be0c3523de8 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:40.696054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56127bc-eae0-4965-be32-1d7f99394396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:40.696635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56127bc-eae0-4965-be32-1d7f99394396 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.696913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56127bc-eae0-4965-be32-1d7f99394396 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.697183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56127bc-eae0-4965-be32-1d7f99394396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.789582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56127bc-eae0-4965-be32-1d7f99394396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:40.793059 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56127bc-eae0-4965-be32-1d7f99394396 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:40.801545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d7863-b8b8-4af4-8b0a-6f939fb28427 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:40.817705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:40.856247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:40.856598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:40.856829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:40.857104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:40.857920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] RBAC: Adding query filter params (role.id=b806354bf1c840cb9329d2e65a7576f2, user.id=f4690ebf76a9492cb10dc97d86896b63, scope.system=all) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:40.858224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:40.860259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:40.877215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 184/367] 10.222.0.109 () {56 vars in 1313 bytes} [Thu Aug 4 22:07:40 2022] GET /identity/v3/role_assignments?user.id=f4690ebf76a9492cb10dc97d86896b63&scope.system=all&role.id=b806354bf1c840cb9329d2e65a7576f2 => generated 512 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:40.878247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:40.948033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:40.953200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 184/368] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:40 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:40.954415 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56127bc-eae0-4965-be32-1d7f99394396 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.035433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be9690-bf96-46cb-b730-c75d73f2428b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.084223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aebf5-9a0c-4b42-9624-6e95a1f775aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.085514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aebf5-9a0c-4b42-9624-6e95a1f775aa None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.085937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aebf5-9a0c-4b42-9624-6e95a1f775aa None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.086214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aebf5-9a0c-4b42-9624-6e95a1f775aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.162346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aebf5-9a0c-4b42-9624-6e95a1f775aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.164777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aebf5-9a0c-4b42-9624-6e95a1f775aa None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.181750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56127bc-eae0-4965-be32-1d7f99394396 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.201913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:41.202532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:41.226898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.228020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.228495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.228959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.230013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:41.230607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.232170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.243027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 185/369] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:40 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:41.244301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.278101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 185/370] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:40 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:41.278965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aebf5-9a0c-4b42-9624-6e95a1f775aa None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.357853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bf5a7-82cd-44d4-a2e5-c6ad259e7ed3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.389317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aebf5-9a0c-4b42-9624-6e95a1f775aa None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.396336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a60cd-74b3-4eb5-be70-cfa897d62e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.397988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a60cd-74b3-4eb5-be70-cfa897d62e0f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.398466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a60cd-74b3-4eb5-be70-cfa897d62e0f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.398989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a60cd-74b3-4eb5-be70-cfa897d62e0f None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.400483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a60cd-74b3-4eb5-be70-cfa897d62e0f None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.402308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a60cd-74b3-4eb5-be70-cfa897d62e0f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.409089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 186/371] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:41.410420 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a60cd-74b3-4eb5-be70-cfa897d62e0f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.431836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246aae-69d3-4af3-b6b9-eb0c5f354d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.432560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246aae-69d3-4af3-b6b9-eb0c5f354d0b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.432749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246aae-69d3-4af3-b6b9-eb0c5f354d0b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.433018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246aae-69d3-4af3-b6b9-eb0c5f354d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.435905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:41.474689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:41.508019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d246aae-69d3-4af3-b6b9-eb0c5f354d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.509625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d246aae-69d3-4af3-b6b9-eb0c5f354d0b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.535400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a60cd-74b3-4eb5-be70-cfa897d62e0f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.573892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4a6cc-a574-4fb3-b2d5-50ac44a8b8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.574922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4a6cc-a574-4fb3-b2d5-50ac44a8b8b7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.575394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4a6cc-a574-4fb3-b2d5-50ac44a8b8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.575858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4a6cc-a574-4fb3-b2d5-50ac44a8b8b7 None admin] PATH_INFO: `/v3/projects/ee8a7cb4ed304a5598e10ca71a963013` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.577200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4a6cc-a574-4fb3-b2d5-50ac44a8b8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8a7cb4ed304a5598e10ca71a963013)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.579050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4a6cc-a574-4fb3-b2d5-50ac44a8b8b7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.581502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 187/372] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/projects/ee8a7cb4ed304a5598e10ca71a963013 => generated 345 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:41.582694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4a6cc-a574-4fb3-b2d5-50ac44a8b8b7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.618285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 186/373] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:41.619219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246aae-69d3-4af3-b6b9-eb0c5f354d0b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.655281 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:41.699584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4a6cc-a574-4fb3-b2d5-50ac44a8b8b7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.717862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:41.728757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246aae-69d3-4af3-b6b9-eb0c5f354d0b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.740316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a17f3-d8ec-4aa1-abdc-d55cb2098a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.742410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a17f3-d8ec-4aa1-abdc-d55cb2098a36 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.742932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a17f3-d8ec-4aa1-abdc-d55cb2098a36 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.743393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a17f3-d8ec-4aa1-abdc-d55cb2098a36 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.744931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a17f3-d8ec-4aa1-abdc-d55cb2098a36 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.746607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a17f3-d8ec-4aa1-abdc-d55cb2098a36 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.753550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 188/374] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:41.754906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a17f3-d8ec-4aa1-abdc-d55cb2098a36 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.768849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34034c0d-947a-4b1a-91ef-73897d43c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.769801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34034c0d-947a-4b1a-91ef-73897d43c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.770060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34034c0d-947a-4b1a-91ef-73897d43c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.770334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34034c0d-947a-4b1a-91ef-73897d43c7f6 None admin] PATH_INFO: `/v3/groups/c6c89047089442ed811ba216f2919811` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.779565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34034c0d-947a-4b1a-91ef-73897d43c7f6 None admin] RBAC: Authorizing `identity:get_group(group_id=c6c89047089442ed811ba216f2919811)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.781245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34034c0d-947a-4b1a-91ef-73897d43c7f6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.782764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 187/375] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/groups/c6c89047089442ed811ba216f2919811 => generated 230 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:41.783540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34034c0d-947a-4b1a-91ef-73897d43c7f6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.884967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a17f3-d8ec-4aa1-abdc-d55cb2098a36 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.914413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:41.921467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34034c0d-947a-4b1a-91ef-73897d43c7f6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:41.937734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0fa11-8412-476d-9063-de04fd8a4c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.938365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0fa11-8412-476d-9063-de04fd8a4c9b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.939550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0fa11-8412-476d-9063-de04fd8a4c9b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.939550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0fa11-8412-476d-9063-de04fd8a4c9b None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.940231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0fa11-8412-476d-9063-de04fd8a4c9b None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.942509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0fa11-8412-476d-9063-de04fd8a4c9b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.943380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 189/376] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:41.945174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb41c2-0040-4411-8b0d-950a887c4854 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.945479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb41c2-0040-4411-8b0d-950a887c4854 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.945757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb41c2-0040-4411-8b0d-950a887c4854 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.947368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 190/377] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:41.948104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb41c2-0040-4411-8b0d-950a887c4854 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:41.971954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:41.972948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:41.973615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:41.973859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:41.974642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] RBAC: Adding query filter params (group.id=897a344a4ae4412fb3c689cbce72ce41, role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=62daff6eeea54042a2879442094ef1b9) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:41.974947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:41.976679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:41.987721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 188/378] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/role_assignments?group.id=897a344a4ae4412fb3c689cbce72ce41&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 261 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:41.989321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:42.020290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:42.096943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb41c2-0040-4411-8b0d-950a887c4854 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:42.126048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbeaba9-c1ae-44a1-bc4f-b179ddf6066e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:42.149557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057eee6-d9fd-46a3-906c-517a17ee8fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:42.150343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057eee6-d9fd-46a3-906c-517a17ee8fbd None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.150730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057eee6-d9fd-46a3-906c-517a17ee8fbd None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.151118 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057eee6-d9fd-46a3-906c-517a17ee8fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.152027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057eee6-d9fd-46a3-906c-517a17ee8fbd None admin] RBAC: Adding query filter params (group.id=c6c89047089442ed811ba216f2919811, role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=ee8a7cb4ed304a5598e10ca71a963013) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:42.152444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057eee6-d9fd-46a3-906c-517a17ee8fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:42.157204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057eee6-d9fd-46a3-906c-517a17ee8fbd None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:42.164899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 191/379] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/role_assignments?group.id=c6c89047089442ed811ba216f2919811&scope.project.id=ee8a7cb4ed304a5598e10ca71a963013&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 261 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:42.166451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f5a67-3d48-47e6-bd67-cadf9e53b305 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.166799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f5a67-3d48-47e6-bd67-cadf9e53b305 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.167080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f5a67-3d48-47e6-bd67-cadf9e53b305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.176720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9834a-0a24-4b20-a169-6f42b5227688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:42.177393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9834a-0a24-4b20-a169-6f42b5227688 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.177694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9834a-0a24-4b20-a169-6f42b5227688 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.177971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9834a-0a24-4b20-a169-6f42b5227688 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.179998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9834a-0a24-4b20-a169-6f42b5227688 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:42.181526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9834a-0a24-4b20-a169-6f42b5227688 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:42.184015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 189/380] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:41 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:42.191227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9834a-0a24-4b20-a169-6f42b5227688 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:42.196083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-180f5a67-3d48-47e6-bd67-cadf9e53b305 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:42.197335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-180f5a67-3d48-47e6-bd67-cadf9e53b305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:42.234719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:42.289330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 192/381] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:42.338435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9834a-0a24-4b20-a169-6f42b5227688 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:42.358542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f5a67-3d48-47e6-bd67-cadf9e53b305 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:42.384701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50831b84-ebc7-49eb-85b4-eb247fa46045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:42.385345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831b84-ebc7-49eb-85b4-eb247fa46045 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.385652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831b84-ebc7-49eb-85b4-eb247fa46045 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.385936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831b84-ebc7-49eb-85b4-eb247fa46045 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.386738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50831b84-ebc7-49eb-85b4-eb247fa46045 None admin] RBAC: Adding query filter params (group.id=897a344a4ae4412fb3c689cbce72ce41, role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=62daff6eeea54042a2879442094ef1b9) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:42.387048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50831b84-ebc7-49eb-85b4-eb247fa46045 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:42.389280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50831b84-ebc7-49eb-85b4-eb247fa46045 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:42.396920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 190/382] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:42 2022] GET /identity/v3/role_assignments?group.id=897a344a4ae4412fb3c689cbce72ce41&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=8f997477100248a9ace1cc28831caaac => generated 261 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:42.436077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:42.490546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f5a67-3d48-47e6-bd67-cadf9e53b305 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:42.513695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858a946-e09e-4d9e-88d0-20cd450432f9 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.514169 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858a946-e09e-4d9e-88d0-20cd450432f9 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.514380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858a946-e09e-4d9e-88d0-20cd450432f9 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.516135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 191/383] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:42 2022] GET /identity => generated 271 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:42.523576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53ce1-77c6-43c9-9f69-08000137d5f1 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.523880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53ce1-77c6-43c9-9f69-08000137d5f1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.524136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53ce1-77c6-43c9-9f69-08000137d5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.533081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0fe83-6b1d-43cc-bf22-cced43180e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:42.534134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0fe83-6b1d-43cc-bf22-cced43180e82 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.534639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0fe83-6b1d-43cc-bf22-cced43180e82 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.535103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0fe83-6b1d-43cc-bf22-cced43180e82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.548293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0fe83-6b1d-43cc-bf22-cced43180e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-c3f0fe83-6b1d-43cc-bf22-cced43180e82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:42.554301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:42.555285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:42.556051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:42.562548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-fca53ce1-77c6-43c9-9f69-08000137d5f1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:42.562918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 193/384] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:42.563788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fca53ce1-77c6-43c9-9f69-08000137d5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:42.565263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0fe83-6b1d-43cc-bf22-cced43180e82 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:42.673607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0fe83-6b1d-43cc-bf22-cced43180e82 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:42.683760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:42.713129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:42.714582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.715053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.715531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.716467 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:42.716960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:42.718612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:42.733285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 194/385] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:42 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:42.734949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:42.743333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 192/386] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:42.775333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca53ce1-77c6-43c9-9f69-08000137d5f1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:42.827972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:42.908614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32efdf-a1a9-4258-bc2b-dd2c40a16f95 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:42.937607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca53ce1-77c6-43c9-9f69-08000137d5f1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:42.981126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a4f4e-c49e-45cd-84c9-57c00c84c69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:42.982230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a4f4e-c49e-45cd-84c9-57c00c84c69f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:42.982702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a4f4e-c49e-45cd-84c9-57c00c84c69f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:42.983230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a4f4e-c49e-45cd-84c9-57c00c84c69f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:42.995066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a4f4e-c49e-45cd-84c9-57c00c84c69f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-1d8a4f4e-c49e-45cd-84c9-57c00c84c69f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:43.001043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:43.001952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:43.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:43.009652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 193/387] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:43.012376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a4f4e-c49e-45cd-84c9-57c00c84c69f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:43.034730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b5b7-4b76-4bfa-86b8-91ae6ec9c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:43.035413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b5b7-4b76-4bfa-86b8-91ae6ec9c5e0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:43.035695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b5b7-4b76-4bfa-86b8-91ae6ec9c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:43.035969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b5b7-4b76-4bfa-86b8-91ae6ec9c5e0 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:43.037469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2b5b7-4b76-4bfa-86b8-91ae6ec9c5e0 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:43.038696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2b5b7-4b76-4bfa-86b8-91ae6ec9c5e0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:43.040634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 195/388] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:42 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:43.058452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b5b7-4b76-4bfa-86b8-91ae6ec9c5e0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:43.127786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:43.142506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a4f4e-c49e-45cd-84c9-57c00c84c69f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:43.188942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:43.189600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:43.189876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:43.190146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:43.193060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:43.193368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:43.195510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:43.203395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:43.222840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 194/389] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:43 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:43.239269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:43.329175 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b5b7-4b76-4bfa-86b8-91ae6ec9c5e0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:43.380311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f68a50-f8b1-4acb-9bc2-c095df06d9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:43.380929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68a50-f8b1-4acb-9bc2-c095df06d9b9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:43.381179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68a50-f8b1-4acb-9bc2-c095df06d9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:43.381723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68a50-f8b1-4acb-9bc2-c095df06d9b9 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:43.382939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f68a50-f8b1-4acb-9bc2-c095df06d9b9 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:43.384385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f68a50-f8b1-4acb-9bc2-c095df06d9b9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:43.387065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 196/390] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:43 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:43.393662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f68a50-f8b1-4acb-9bc2-c095df06d9b9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:43.464354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7932-0205-418f-af48-f053d05843a8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:43.555819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f68a50-f8b1-4acb-9bc2-c095df06d9b9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:43.558418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:43.562534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c6140-4cc0-47b5-8a78-260b2bf9b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:43.563419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c6140-4cc0-47b5-8a78-260b2bf9b226 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:43.565599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c6140-4cc0-47b5-8a78-260b2bf9b226 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:43.565599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c6140-4cc0-47b5-8a78-260b2bf9b226 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:43.570239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c6140-4cc0-47b5-8a78-260b2bf9b226 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:43.571802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c6140-4cc0-47b5-8a78-260b2bf9b226 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:43.573508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 195/391] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:43 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:43.579371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c6140-4cc0-47b5-8a78-260b2bf9b226 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:43.600089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce969e7c-ff81-43ca-8a95-c161d96b87f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:43.601288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce969e7c-ff81-43ca-8a95-c161d96b87f3 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:43.601595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce969e7c-ff81-43ca-8a95-c161d96b87f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:43.601867 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce969e7c-ff81-43ca-8a95-c161d96b87f3 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1/groups/897a344a4ae4412fb3c689cbce72ce41/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:43.606312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce969e7c-ff81-43ca-8a95-c161d96b87f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3aa2c7542ee14875b5538c96394941d1, group_id=897a344a4ae4412fb3c689cbce72ce41, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:43.607760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce969e7c-ff81-43ca-8a95-c161d96b87f3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:43.619216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:43.636189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 197/392] 10.222.0.109 () {58 vars in 1447 bytes} [Thu Aug 4 22:07:43 2022] PUT /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1/groups/897a344a4ae4412fb3c689cbce72ce41/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:43.731952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c6140-4cc0-47b5-8a78-260b2bf9b226 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:43.776004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48fa3ea-82f5-4ff9-a9b6-89f5dfa513d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:43.776677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fa3ea-82f5-4ff9-a9b6-89f5dfa513d7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:43.776959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fa3ea-82f5-4ff9-a9b6-89f5dfa513d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:43.777235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fa3ea-82f5-4ff9-a9b6-89f5dfa513d7 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:43.778555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48fa3ea-82f5-4ff9-a9b6-89f5dfa513d7 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:43.780125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48fa3ea-82f5-4ff9-a9b6-89f5dfa513d7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:43.782228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 196/393] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:43 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:43.790897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce969e7c-ff81-43ca-8a95-c161d96b87f3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:43.921724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce969e7c-ff81-43ca-8a95-c161d96b87f3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:43.963051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f334e3-3cb2-44db-be32-37da28dbcf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:43.964098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f334e3-3cb2-44db-be32-37da28dbcf2f None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:43.964629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f334e3-3cb2-44db-be32-37da28dbcf2f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:43.965132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f334e3-3cb2-44db-be32-37da28dbcf2f None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1/groups/897a344a4ae4412fb3c689cbce72ce41/roles/8f997477100248a9ace1cc28831caaac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:43.967649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f334e3-3cb2-44db-be32-37da28dbcf2f None admin] RBAC: Authorizing `identity:create_grant(project_id=3aa2c7542ee14875b5538c96394941d1, group_id=897a344a4ae4412fb3c689cbce72ce41, role_id=8f997477100248a9ace1cc28831caaac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:43.969344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f334e3-3cb2-44db-be32-37da28dbcf2f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:43.988101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 198/394] 10.222.0.109 () {58 vars in 1447 bytes} [Thu Aug 4 22:07:43 2022] PUT /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1/groups/897a344a4ae4412fb3c689cbce72ce41/roles/8f997477100248a9ace1cc28831caaac => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:44.616595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ad2e-fdad-490a-8b61-e22de9015bcf None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:44.617148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ad2e-fdad-490a-8b61-e22de9015bcf None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:44.617148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ad2e-fdad-490a-8b61-e22de9015bcf None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:44.618557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 197/395] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:44.624772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8e229-1de3-4a1d-b6ff-4c3d2beef609 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:44.625058 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8e229-1de3-4a1d-b6ff-4c3d2beef609 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:44.625332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8e229-1de3-4a1d-b6ff-4c3d2beef609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:44.654484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-c0a8e229-1de3-4a1d-b6ff-4c3d2beef609 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:44.655704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c0a8e229-1de3-4a1d-b6ff-4c3d2beef609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:44.675679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:44.742978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 199/396] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:44.778587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408ad2e-fdad-490a-8b61-e22de9015bcf None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:44.904266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:44.923633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569177f-68ef-4fd5-ad9c-3d5792ae613d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:44.924604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569177f-68ef-4fd5-ad9c-3d5792ae613d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:44.925112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569177f-68ef-4fd5-ad9c-3d5792ae613d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:44.927099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 200/397] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:44 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:44.932992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408ad2e-fdad-490a-8b61-e22de9015bcf None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:44.933820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65a9ba-98a9-4650-9be6-2f9ab3d8cd65 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:44.934315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65a9ba-98a9-4650-9be6-2f9ab3d8cd65 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:44.934788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65a9ba-98a9-4650-9be6-2f9ab3d8cd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:44.954890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3b65a9ba-98a9-4650-9be6-2f9ab3d8cd65 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:44.956517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3b65a9ba-98a9-4650-9be6-2f9ab3d8cd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:45.004099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270869-82b7-4b5e-bc68-fc85fb190bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.005226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270869-82b7-4b5e-bc68-fc85fb190bb7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.005732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270869-82b7-4b5e-bc68-fc85fb190bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.006763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270869-82b7-4b5e-bc68-fc85fb190bb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.029961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70270869-82b7-4b5e-bc68-fc85fb190bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-70270869-82b7-4b5e-bc68-fc85fb190bb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:45.037316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:45.039065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:45.039925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:45.046752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 198/398] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:45.054353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270869-82b7-4b5e-bc68-fc85fb190bb7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.083742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 201/399] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:45.085744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437d6a9-69bb-42de-8ff1-99b37e98ccf7 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.086099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437d6a9-69bb-42de-8ff1-99b37e98ccf7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.086439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437d6a9-69bb-42de-8ff1-99b37e98ccf7 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.088035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 202/400] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:45.095892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd8ad3-d881-45d9-b2dd-f1f448baf6d8 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.095892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd8ad3-d881-45d9-b2dd-f1f448baf6d8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.095892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd8ad3-d881-45d9-b2dd-f1f448baf6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.130011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-afdd8ad3-d881-45d9-b2dd-f1f448baf6d8 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:45.132006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-afdd8ad3-d881-45d9-b2dd-f1f448baf6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:45.175667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270869-82b7-4b5e-bc68-fc85fb190bb7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.216114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.217030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.217519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.217980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.218150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 203/401] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:45.219091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd8ad3-d881-45d9-b2dd-f1f448baf6d8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.219203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] RBAC: Adding query filter params (name=member) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:45.219677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:45.221542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:45.232924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 199/402] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:45.235457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:45.237891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.243134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:45.342377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd8ad3-d881-45d9-b2dd-f1f448baf6d8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.366655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35a270-611a-41dc-a187-0e955b5d0024 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.380490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bff5d-8d55-4167-a5f0-65fe37d79d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.382121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bff5d-8d55-4167-a5f0-65fe37d79d6c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.382611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bff5d-8d55-4167-a5f0-65fe37d79d6c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.383070 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bff5d-8d55-4167-a5f0-65fe37d79d6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.394876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bff5d-8d55-4167-a5f0-65fe37d79d6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-c98bff5d-8d55-4167-a5f0-65fe37d79d6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:45.400681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:45.401586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:45.402524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:45.404704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 204/403] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:45.405761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bff5d-8d55-4167-a5f0-65fe37d79d6c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.407441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b9453-6b40-44eb-bde9-4c5a31ee0e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.408088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9453-6b40-44eb-bde9-4c5a31ee0e19 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.408366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9453-6b40-44eb-bde9-4c5a31ee0e19 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.408639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9453-6b40-44eb-bde9-4c5a31ee0e19 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.410122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b9453-6b40-44eb-bde9-4c5a31ee0e19 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:45.412668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b9453-6b40-44eb-bde9-4c5a31ee0e19 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:45.414674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 200/404] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:45.417723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b9453-6b40-44eb-bde9-4c5a31ee0e19 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.493583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:45.524144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bff5d-8d55-4167-a5f0-65fe37d79d6c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.524394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:45.565147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b9453-6b40-44eb-bde9-4c5a31ee0e19 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.567297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450a0a0-3a1b-48fc-aa63-2eef086ae2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.568801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450a0a0-3a1b-48fc-aa63-2eef086ae2c8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.569155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450a0a0-3a1b-48fc-aa63-2eef086ae2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.569539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450a0a0-3a1b-48fc-aa63-2eef086ae2c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.583528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450a0a0-3a1b-48fc-aa63-2eef086ae2c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-8450a0a0-3a1b-48fc-aa63-2eef086ae2c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:45.589353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:45.590306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:45.591087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:45.593986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 205/405] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:45.595009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450a0a0-3a1b-48fc-aa63-2eef086ae2c8 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.606197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.606818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.607097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.607368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.608077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:45.608378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:45.609800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:45.619418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 201/406] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:45.620951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.727235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450a0a0-3a1b-48fc-aa63-2eef086ae2c8 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.746200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a795d4-7944-48eb-8c87-09f194d61cd7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.778740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2efb8-e157-4f6e-a2dc-604b2c166a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.779880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2efb8-e157-4f6e-a2dc-604b2c166a52 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.780300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2efb8-e157-4f6e-a2dc-604b2c166a52 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.780728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2efb8-e157-4f6e-a2dc-604b2c166a52 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.782224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2efb8-e157-4f6e-a2dc-604b2c166a52 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:45.783507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.783983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2efb8-e157-4f6e-a2dc-604b2c166a52 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:45.784326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.784816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.785194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.786039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:45.786526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:45.786902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 206/407] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:45.787909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2efb8-e157-4f6e-a2dc-604b2c166a52 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.788140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:45.790163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:45.793059 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:45.804026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 202/408] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:45.806888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.910630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2efb8-e157-4f6e-a2dc-604b2c166a52 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.936854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be2c77-b874-4e1a-a523-7622a97a30a6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:45.947939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82741ee4-fb75-4754-ad28-015ec1551b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.948478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741ee4-fb75-4754-ad28-015ec1551b65 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.948764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741ee4-fb75-4754-ad28-015ec1551b65 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.949041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741ee4-fb75-4754-ad28-015ec1551b65 None admin] PATH_INFO: `/v3/groups/c6c89047089442ed811ba216f2919811` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.950259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82741ee4-fb75-4754-ad28-015ec1551b65 None admin] RBAC: Authorizing `identity:get_group(group_id=c6c89047089442ed811ba216f2919811)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:45.951722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82741ee4-fb75-4754-ad28-015ec1551b65 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:45.957851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 207/409] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/groups/c6c89047089442ed811ba216f2919811 => generated 230 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:45.958762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82741ee4-fb75-4754-ad28-015ec1551b65 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:45.981635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22aedce-0367-46ab-bdac-700273e60174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:45.982578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22aedce-0367-46ab-bdac-700273e60174 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:45.983463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22aedce-0367-46ab-bdac-700273e60174 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:45.983890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22aedce-0367-46ab-bdac-700273e60174 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9/groups/897a344a4ae4412fb3c689cbce72ce41/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:45.986961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22aedce-0367-46ab-bdac-700273e60174 None admin] RBAC: Authorizing `identity:create_grant(project_id=62daff6eeea54042a2879442094ef1b9, group_id=897a344a4ae4412fb3c689cbce72ce41, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:45.989182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22aedce-0367-46ab-bdac-700273e60174 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:46.010255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 203/410] 10.222.0.109 () {58 vars in 1447 bytes} [Thu Aug 4 22:07:45 2022] PUT /identity/v3/projects/62daff6eeea54042a2879442094ef1b9/groups/897a344a4ae4412fb3c689cbce72ce41/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:46.011661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22aedce-0367-46ab-bdac-700273e60174 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:46.041719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:46.089807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:46.117762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82741ee4-fb75-4754-ad28-015ec1551b65 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:46.149865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22aedce-0367-46ab-bdac-700273e60174 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:46.184678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de978-0981-4604-a5cc-9888fbdc519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:46.185645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de978-0981-4604-a5cc-9888fbdc519c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.186083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de978-0981-4604-a5cc-9888fbdc519c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.186511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de978-0981-4604-a5cc-9888fbdc519c None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.188208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852de978-0981-4604-a5cc-9888fbdc519c None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:46.190406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852de978-0981-4604-a5cc-9888fbdc519c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:46.192599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 208/411] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:45 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:46.199314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de978-0981-4604-a5cc-9888fbdc519c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:46.233389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcf19a-ee88-4aa0-a4c6-0b275a655a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:46.234173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcf19a-ee88-4aa0-a4c6-0b275a655a5b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.234457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcf19a-ee88-4aa0-a4c6-0b275a655a5b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.234733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcf19a-ee88-4aa0-a4c6-0b275a655a5b None admin] PATH_INFO: `/v3/projects/ee8a7cb4ed304a5598e10ca71a963013` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.235933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dcf19a-ee88-4aa0-a4c6-0b275a655a5b None admin] RBAC: Authorizing `identity:get_project(project_id=ee8a7cb4ed304a5598e10ca71a963013)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:46.250489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dcf19a-ee88-4aa0-a4c6-0b275a655a5b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:46.252357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 204/412] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:46 2022] GET /identity/v3/projects/ee8a7cb4ed304a5598e10ca71a963013 => generated 345 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:46.262582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcf19a-ee88-4aa0-a4c6-0b275a655a5b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:46.331806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de978-0981-4604-a5cc-9888fbdc519c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:46.380982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af70322-98c6-4018-b63c-f9de8e7c980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:46.382235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af70322-98c6-4018-b63c-f9de8e7c980e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.382515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af70322-98c6-4018-b63c-f9de8e7c980e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.383347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af70322-98c6-4018-b63c-f9de8e7c980e None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.385904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcf19a-ee88-4aa0-a4c6-0b275a655a5b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:46.386042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af70322-98c6-4018-b63c-f9de8e7c980e None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:46.389587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af70322-98c6-4018-b63c-f9de8e7c980e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:46.391783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 209/413] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:46 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:46.393047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3732b-6b6f-4ea3-bca5-0527e57f3aec None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.393332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3732b-6b6f-4ea3-bca5-0527e57f3aec None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.393629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3732b-6b6f-4ea3-bca5-0527e57f3aec None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.397547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 210/414] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:46.398307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3732b-6b6f-4ea3-bca5-0527e57f3aec None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:46.411392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:46.417798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:46.431026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d7196-99fc-4025-99a0-19aac732327a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:46.431824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d7196-99fc-4025-99a0-19aac732327a None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.432109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d7196-99fc-4025-99a0-19aac732327a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.432388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d7196-99fc-4025-99a0-19aac732327a None admin] PATH_INFO: `/v3/projects/ee8a7cb4ed304a5598e10ca71a963013/groups/c6c89047089442ed811ba216f2919811/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.434509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d7196-99fc-4025-99a0-19aac732327a None admin] RBAC: Authorizing `identity:create_grant(project_id=ee8a7cb4ed304a5598e10ca71a963013, group_id=c6c89047089442ed811ba216f2919811, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:46.436093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d7196-99fc-4025-99a0-19aac732327a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:46.451283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 205/415] 10.222.0.109 () {58 vars in 1447 bytes} [Thu Aug 4 22:07:46 2022] PUT /identity/v3/projects/ee8a7cb4ed304a5598e10ca71a963013/groups/c6c89047089442ed811ba216f2919811/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:46.458880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258b6c-9334-4601-a781-d12b0467429f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.459198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258b6c-9334-4601-a781-d12b0467429f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.459482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258b6c-9334-4601-a781-d12b0467429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.484154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-56258b6c-9334-4601-a781-d12b0467429f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:46.485623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-56258b6c-9334-4601-a781-d12b0467429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:46.529911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3732b-6b6f-4ea3-bca5-0527e57f3aec None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:46.571512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 206/416] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:46.571512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff95b-6ef3-40fc-bdf3-c32866df3ceb None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.571512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff95b-6ef3-40fc-bdf3-c32866df3ceb None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.571512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff95b-6ef3-40fc-bdf3-c32866df3ceb None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.571512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 207/417] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:46.575765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a6f38-003f-43d5-b609-218f3f6fee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:46.576460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6f38-003f-43d5-b609-218f3f6fee6b None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.576740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6f38-003f-43d5-b609-218f3f6fee6b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.577013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6f38-003f-43d5-b609-218f3f6fee6b None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9/groups/897a344a4ae4412fb3c689cbce72ce41/roles/8f997477100248a9ace1cc28831caaac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.578932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a95ff-35d8-4698-be7e-6a97428719d4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.579217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a95ff-35d8-4698-be7e-6a97428719d4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.579487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a95ff-35d8-4698-be7e-6a97428719d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.585634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a6f38-003f-43d5-b609-218f3f6fee6b None admin] RBAC: Authorizing `identity:create_grant(project_id=62daff6eeea54042a2879442094ef1b9, group_id=897a344a4ae4412fb3c689cbce72ce41, role_id=8f997477100248a9ace1cc28831caaac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:46.587106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a6f38-003f-43d5-b609-218f3f6fee6b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:46.608229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b61a95ff-35d8-4698-be7e-6a97428719d4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:46.609517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b61a95ff-35d8-4698-be7e-6a97428719d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:46.610504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 211/418] 10.222.0.109 () {58 vars in 1447 bytes} [Thu Aug 4 22:07:46 2022] PUT /identity/v3/projects/62daff6eeea54042a2879442094ef1b9/groups/897a344a4ae4412fb3c689cbce72ce41/roles/8f997477100248a9ace1cc28831caaac => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:07:46.611803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a6f38-003f-43d5-b609-218f3f6fee6b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:46.700165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:46.702082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 208/419] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:46.731464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a95ff-35d8-4698-be7e-6a97428719d4 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:46.783868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a6f38-003f-43d5-b609-218f3f6fee6b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:46.799648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:46.836883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254b551-3571-4a73-8b96-2aeca11db579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:46.837991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254b551-3571-4a73-8b96-2aeca11db579 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.839031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254b551-3571-4a73-8b96-2aeca11db579 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.839514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254b551-3571-4a73-8b96-2aeca11db579 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.855105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254b551-3571-4a73-8b96-2aeca11db579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-5254b551-3571-4a73-8b96-2aeca11db579 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:46.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:46.864143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:46.865132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:46.871518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 212/420] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:46.873159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254b551-3571-4a73-8b96-2aeca11db579 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:46.878396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a95ff-35d8-4698-be7e-6a97428719d4 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:46.940112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced24b5-ceb2-49d7-bdcb-71ddb5672706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:46.941214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced24b5-ceb2-49d7-bdcb-71ddb5672706 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:46.941706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced24b5-ceb2-49d7-bdcb-71ddb5672706 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:46.942210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced24b5-ceb2-49d7-bdcb-71ddb5672706 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:46.959227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced24b5-ceb2-49d7-bdcb-71ddb5672706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-8ced24b5-ceb2-49d7-bdcb-71ddb5672706 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:46.964339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:46.965329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:46.966544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 209/421] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:46.971494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced24b5-ceb2-49d7-bdcb-71ddb5672706 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:47.073088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254b551-3571-4a73-8b96-2aeca11db579 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:47.129204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:47.146552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:47.147555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:47.147837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:47.148110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:47.148809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:47.149106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:47.150504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:47.164436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced24b5-ceb2-49d7-bdcb-71ddb5672706 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:47.168315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 213/422] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:46 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:47.173889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:47.175312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:47.230027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:47.230645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:47.230919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:47.231231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:47.231946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:47.232243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:47.233644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:47.255459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 210/423] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:46 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:47.261846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:47.317949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5853b-c72b-463e-a96a-98c1af458bc6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:47.358860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211818ea-ed07-48bc-89e8-21a3d4a64db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:47.359916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211818ea-ed07-48bc-89e8-21a3d4a64db4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:47.360348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211818ea-ed07-48bc-89e8-21a3d4a64db4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:47.361008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211818ea-ed07-48bc-89e8-21a3d4a64db4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:47.416883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3534192-a9f4-4842-8713-ed0d6381d6ad None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:47.453344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363bcd9-3840-4938-935c-a7c0e9c6e038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:47.454144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363bcd9-3840-4938-935c-a7c0e9c6e038 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:47.454557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363bcd9-3840-4938-935c-a7c0e9c6e038 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:47.454929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363bcd9-3840-4938-935c-a7c0e9c6e038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:47.489834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:47.526872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363bcd9-3840-4938-935c-a7c0e9c6e038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:47.528432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363bcd9-3840-4938-935c-a7c0e9c6e038 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:47.534427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:47.572376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211818ea-ed07-48bc-89e8-21a3d4a64db4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:47.573884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211818ea-ed07-48bc-89e8-21a3d4a64db4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:47.635636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 211/424] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 22:07:47.641401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363bcd9-3840-4938-935c-a7c0e9c6e038 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:47.713767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:47.747095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363bcd9-3840-4938-935c-a7c0e9c6e038 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:47.783161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26167-e322-4e21-b896-10071a554b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:47.784111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26167-e322-4e21-b896-10071a554b63 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:47.786746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26167-e322-4e21-b896-10071a554b63 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:47.787183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26167-e322-4e21-b896-10071a554b63 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:47.789131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e26167-e322-4e21-b896-10071a554b63 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:47.791705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e26167-e322-4e21-b896-10071a554b63 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:47.796057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 212/425] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:47 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:47.802578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26167-e322-4e21-b896-10071a554b63 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:47.868751 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 214/426] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:47.874099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211818ea-ed07-48bc-89e8-21a3d4a64db4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:47.981623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:47.997844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211818ea-ed07-48bc-89e8-21a3d4a64db4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:48.022599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26167-e322-4e21-b896-10071a554b63 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:48.060118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21569420-2c07-4b00-ac57-3d2845892c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:48.061200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21569420-2c07-4b00-ac57-3d2845892c4b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.061735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21569420-2c07-4b00-ac57-3d2845892c4b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.062289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21569420-2c07-4b00-ac57-3d2845892c4b None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.063274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659e0df-4ad0-4a4d-a3b7-fea54b454791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:48.063955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e0df-4ad0-4a4d-a3b7-fea54b454791 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.064239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e0df-4ad0-4a4d-a3b7-fea54b454791 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.064514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e0df-4ad0-4a4d-a3b7-fea54b454791 None admin] PATH_INFO: `/v3/projects/3aa2c7542ee14875b5538c96394941d1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.064731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21569420-2c07-4b00-ac57-3d2845892c4b None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:48.066552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21569420-2c07-4b00-ac57-3d2845892c4b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:48.069431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 213/427] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:47 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:48.073474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659e0df-4ad0-4a4d-a3b7-fea54b454791 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa2c7542ee14875b5538c96394941d1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:48.073474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659e0df-4ad0-4a4d-a3b7-fea54b454791 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:48.074001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 215/428] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:47 2022] GET /identity/v3/projects/3aa2c7542ee14875b5538c96394941d1 => generated 299 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:48.076235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21569420-2c07-4b00-ac57-3d2845892c4b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:48.079167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659e0df-4ad0-4a4d-a3b7-fea54b454791 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:48.079659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:48.202887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21569420-2c07-4b00-ac57-3d2845892c4b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:48.227692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659e0df-4ad0-4a4d-a3b7-fea54b454791 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:48.263379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340381d5-daf4-4a14-832c-8ae5d4ef1179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:48.263949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340381d5-daf4-4a14-832c-8ae5d4ef1179 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.264231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340381d5-daf4-4a14-832c-8ae5d4ef1179 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.264507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340381d5-daf4-4a14-832c-8ae5d4ef1179 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.267050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340381d5-daf4-4a14-832c-8ae5d4ef1179 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:48.268508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340381d5-daf4-4a14-832c-8ae5d4ef1179 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:48.270405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 216/429] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:48 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:48.277278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340381d5-daf4-4a14-832c-8ae5d4ef1179 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:48.278244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c20754-610b-46b9-b2ef-b8bebfaa7393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:48.278866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20754-610b-46b9-b2ef-b8bebfaa7393 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.279146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20754-610b-46b9-b2ef-b8bebfaa7393 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.279418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20754-610b-46b9-b2ef-b8bebfaa7393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.280218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c20754-610b-46b9-b2ef-b8bebfaa7393 None admin] RBAC: Adding query filter params (group.id=897a344a4ae4412fb3c689cbce72ce41, role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=3aa2c7542ee14875b5538c96394941d1) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:48.280529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c20754-610b-46b9-b2ef-b8bebfaa7393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:48.282010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c20754-610b-46b9-b2ef-b8bebfaa7393 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:48.301711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 214/430] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:48 2022] GET /identity/v3/role_assignments?group.id=897a344a4ae4412fb3c689cbce72ce41&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=8f997477100248a9ace1cc28831caaac => generated 613 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:48.317852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:48.399283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340381d5-daf4-4a14-832c-8ae5d4ef1179 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:48.437746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b01af-7fa7-4d96-b081-d40d7c7fb884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:48.438379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b01af-7fa7-4d96-b081-d40d7c7fb884 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.438660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b01af-7fa7-4d96-b081-d40d7c7fb884 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.438936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b01af-7fa7-4d96-b081-d40d7c7fb884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.439731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b01af-7fa7-4d96-b081-d40d7c7fb884 None admin] RBAC: Adding query filter params (group.id=897a344a4ae4412fb3c689cbce72ce41, role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=3aa2c7542ee14875b5538c96394941d1) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:48.440110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b01af-7fa7-4d96-b081-d40d7c7fb884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:48.441715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b01af-7fa7-4d96-b081-d40d7c7fb884 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:48.449485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 217/431] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:48 2022] GET /identity/v3/role_assignments?group.id=897a344a4ae4412fb3c689cbce72ce41&scope.project.id=3aa2c7542ee14875b5538c96394941d1&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 613 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:48.545498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89349abe-5271-4799-9f6e-faac72b051b3 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.545683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89349abe-5271-4799-9f6e-faac72b051b3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.545958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89349abe-5271-4799-9f6e-faac72b051b3 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.547637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 215/432] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:48.554927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b975-7875-44ec-a8c7-89a6c1115e68 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.555268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b975-7875-44ec-a8c7-89a6c1115e68 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.555463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b975-7875-44ec-a8c7-89a6c1115e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.578527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e6d2b975-7875-44ec-a8c7-89a6c1115e68 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:48.579745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e6d2b975-7875-44ec-a8c7-89a6c1115e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:48.675209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 218/433] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:48.699814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89349abe-5271-4799-9f6e-faac72b051b3 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:48.777129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:48.820868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89349abe-5271-4799-9f6e-faac72b051b3 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:48.856601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed2c4d-39b3-4424-a845-d810167977d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:48.857238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed2c4d-39b3-4424-a845-d810167977d0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.857559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed2c4d-39b3-4424-a845-d810167977d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.857834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed2c4d-39b3-4424-a845-d810167977d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.868831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed2c4d-39b3-4424-a845-d810167977d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-dbed2c4d-39b3-4424-a845-d810167977d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:48.874399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:48.875308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:48.876090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 216/434] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:48.881547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2b975-7875-44ec-a8c7-89a6c1115e68 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:48.897364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d11bd-d98a-4087-aa89-df4434e260dc None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.898155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d11bd-d98a-4087-aa89-df4434e260dc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.898615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d11bd-d98a-4087-aa89-df4434e260dc None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.900596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 217/435] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:48 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:48.905932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd859-0937-4f18-981a-eb3bbc06ac70 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:48.906216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd859-0937-4f18-981a-eb3bbc06ac70 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:48.906488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd859-0937-4f18-981a-eb3bbc06ac70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:48.917105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:48.930012 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9d8bd859-0937-4f18-981a-eb3bbc06ac70 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:48.931207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9d8bd859-0937-4f18-981a-eb3bbc06ac70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:49.021452 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 218/436] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:49.040849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2b975-7875-44ec-a8c7-89a6c1115e68 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.048727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bd859-0937-4f18-981a-eb3bbc06ac70 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.090186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40f732-bee6-4cc5-a912-6d287a9aee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.090826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f732-bee6-4cc5-a912-6d287a9aee38 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.091103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f732-bee6-4cc5-a912-6d287a9aee38 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.091371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f732-bee6-4cc5-a912-6d287a9aee38 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.092093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40f732-bee6-4cc5-a912-6d287a9aee38 None admin] RBAC: Adding query filter params (name=member) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:49.092392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40f732-bee6-4cc5-a912-6d287a9aee38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:49.093802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40f732-bee6-4cc5-a912-6d287a9aee38 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:49.104620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 219/437] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:07:48 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:49.106233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485beaa2-fb68-405e-9a97-e8b0162fad1d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.106233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485beaa2-fb68-405e-9a97-e8b0162fad1d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.106401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485beaa2-fb68-405e-9a97-e8b0162fad1d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.107684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 220/438] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:49.113828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485beaa2-fb68-405e-9a97-e8b0162fad1d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.119672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:49.168988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bd859-0937-4f18-981a-eb3bbc06ac70 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.203217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:49.205127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da84bf-d5f1-4dea-8ec4-4cfe126ccf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.205798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da84bf-d5f1-4dea-8ec4-4cfe126ccf1e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.206077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da84bf-d5f1-4dea-8ec4-4cfe126ccf1e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.206349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da84bf-d5f1-4dea-8ec4-4cfe126ccf1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.216816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da84bf-d5f1-4dea-8ec4-4cfe126ccf1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-f4da84bf-d5f1-4dea-8ec4-4cfe126ccf1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:49.221730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:49.222610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:49.223403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 219/439] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:49.224051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485beaa2-fb68-405e-9a97-e8b0162fad1d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.224129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3f6fa-f649-43b1-8ccf-a84c1028944d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.224533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3f6fa-f649-43b1-8ccf-a84c1028944d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.224533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3f6fa-f649-43b1-8ccf-a84c1028944d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.243360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7ac3f6fa-f649-43b1-8ccf-a84c1028944d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:49.244559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7ac3f6fa-f649-43b1-8ccf-a84c1028944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:49.258334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6693e0-b085-45de-8d6d-7d1fe51aa367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.258953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6693e0-b085-45de-8d6d-7d1fe51aa367 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.259233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6693e0-b085-45de-8d6d-7d1fe51aa367 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.259502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6693e0-b085-45de-8d6d-7d1fe51aa367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.309674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 220/440] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:49.310574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3f6fa-f649-43b1-8ccf-a84c1028944d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.325632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6693e0-b085-45de-8d6d-7d1fe51aa367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:49.327082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6693e0-b085-45de-8d6d-7d1fe51aa367 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:49.359384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:49.410411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:49.411707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3f6fa-f649-43b1-8ccf-a84c1028944d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.431168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 221/441] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:49.432251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6693e0-b085-45de-8d6d-7d1fe51aa367 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.446608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.447230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.447508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.447783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.448494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:49.448812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:49.450243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:49.458562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 221/442] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:49.460552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.537507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6693e0-b085-45de-8d6d-7d1fe51aa367 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.560384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e116e-66d8-4c6e-a9cf-cd2c577feccb None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.570021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:49.571402 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe5a1f-1c5d-4beb-9736-91e18732a30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.572097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe5a1f-1c5d-4beb-9736-91e18732a30c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.572379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe5a1f-1c5d-4beb-9736-91e18732a30c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.572651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe5a1f-1c5d-4beb-9736-91e18732a30c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.582643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe5a1f-1c5d-4beb-9736-91e18732a30c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-dafe5a1f-1c5d-4beb-9736-91e18732a30c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:49.588283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:49.589180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:49.590390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 222/443] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:49.591304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe5a1f-1c5d-4beb-9736-91e18732a30c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.600284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4b9c9-fe27-47de-9ee6-ae796a9ffeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.600912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b9c9-fe27-47de-9ee6-ae796a9ffeeb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.601187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b9c9-fe27-47de-9ee6-ae796a9ffeeb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.601485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b9c9-fe27-47de-9ee6-ae796a9ffeeb None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.602666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4b9c9-fe27-47de-9ee6-ae796a9ffeeb None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:49.604095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4b9c9-fe27-47de-9ee6-ae796a9ffeeb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:49.605780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 222/444] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:49.605780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4b9c9-fe27-47de-9ee6-ae796a9ffeeb None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.636471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:49.695583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe5a1f-1c5d-4beb-9736-91e18732a30c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.708147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4b9c9-fe27-47de-9ee6-ae796a9ffeeb None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.729490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2e2a7-d157-441d-910f-8f2b81133a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.730101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2e2a7-d157-441d-910f-8f2b81133a63 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.730373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2e2a7-d157-441d-910f-8f2b81133a63 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.730647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2e2a7-d157-441d-910f-8f2b81133a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.741425 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.742030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.742305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.742576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.743283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:49.743584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:49.744951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:49.752648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 223/445] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:49.753436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.793354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:49.798123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2e2a7-d157-441d-910f-8f2b81133a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:49.799605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2e2a7-d157-441d-910f-8f2b81133a63 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:49.839963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:49.853339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52c25-bd55-4e58-8a36-77bef46b38c6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.886559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abbb69-d278-4b52-8e1b-c3c46bde2b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:49.887120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abbb69-d278-4b52-8e1b-c3c46bde2b04 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:49.887395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abbb69-d278-4b52-8e1b-c3c46bde2b04 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:49.887665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abbb69-d278-4b52-8e1b-c3c46bde2b04 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:49.888803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abbb69-d278-4b52-8e1b-c3c46bde2b04 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:49.890275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abbb69-d278-4b52-8e1b-c3c46bde2b04 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:49.891852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 224/446] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:49.892598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abbb69-d278-4b52-8e1b-c3c46bde2b04 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.903863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 223/447] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:49.903863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2e2a7-d157-441d-910f-8f2b81133a63 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:49.992937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abbb69-d278-4b52-8e1b-c3c46bde2b04 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.996101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2e2a7-d157-441d-910f-8f2b81133a63 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:49.999698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:50.028801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770d21a-cfb9-4b09-ae85-6512243c32a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:50.029013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:50.029432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770d21a-cfb9-4b09-ae85-6512243c32a1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:50.029667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:50.029870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770d21a-cfb9-4b09-ae85-6512243c32a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:50.029958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:50.030208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:50.030299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770d21a-cfb9-4b09-ae85-6512243c32a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:50.030992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] RBAC: Adding query filter params (group.id=897a344a4ae4412fb3c689cbce72ce41, role.id=bad0ac6580d5470fb6dd23a137576b9c, scope.project.id=62daff6eeea54042a2879442094ef1b9) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:50.031301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:50.032734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:50.039309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 224/448] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/role_assignments?group.id=897a344a4ae4412fb3c689cbce72ce41&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=bad0ac6580d5470fb6dd23a137576b9c => generated 613 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:50.040271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:50.043918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:50.124658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770d21a-cfb9-4b09-ae85-6512243c32a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:50.126229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770d21a-cfb9-4b09-ae85-6512243c32a1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:50.156931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2607e7-4f24-41d2-b24a-e5c5f63f9940 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:50.197438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830198c-2933-4839-8b45-4efcfcd29d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:50.198104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830198c-2933-4839-8b45-4efcfcd29d62 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:50.198383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830198c-2933-4839-8b45-4efcfcd29d62 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:50.198654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830198c-2933-4839-8b45-4efcfcd29d62 None admin] PATH_INFO: `/v3/projects/ee8a7cb4ed304a5598e10ca71a963013` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:50.199831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830198c-2933-4839-8b45-4efcfcd29d62 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8a7cb4ed304a5598e10ca71a963013)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:50.201293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830198c-2933-4839-8b45-4efcfcd29d62 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:50.203333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 225/449] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:50 2022] GET /identity/v3/projects/ee8a7cb4ed304a5598e10ca71a963013 => generated 345 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:50.209471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830198c-2933-4839-8b45-4efcfcd29d62 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:50.242509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 225/450] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:49 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:50.247867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770d21a-cfb9-4b09-ae85-6512243c32a1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:50.256385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:50.298713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:50.335008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830198c-2933-4839-8b45-4efcfcd29d62 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:50.377099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770d21a-cfb9-4b09-ae85-6512243c32a1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:50.384645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6abe6b-18fe-4708-af73-166f60139b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:50.385385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6abe6b-18fe-4708-af73-166f60139b46 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:50.385781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6abe6b-18fe-4708-af73-166f60139b46 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:50.386449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6abe6b-18fe-4708-af73-166f60139b46 None admin] PATH_INFO: `/v3/groups/c6c89047089442ed811ba216f2919811` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:50.387550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6abe6b-18fe-4708-af73-166f60139b46 None admin] RBAC: Authorizing `identity:get_group(group_id=c6c89047089442ed811ba216f2919811)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:50.389191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6abe6b-18fe-4708-af73-166f60139b46 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:50.391468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 226/451] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:50 2022] GET /identity/v3/groups/c6c89047089442ed811ba216f2919811 => generated 230 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:50.397433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6abe6b-18fe-4708-af73-166f60139b46 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:50.415871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a23a1b-8650-471e-bd10-7389f72968f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:50.416441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a23a1b-8650-471e-bd10-7389f72968f3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:50.416719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a23a1b-8650-471e-bd10-7389f72968f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:50.416993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a23a1b-8650-471e-bd10-7389f72968f3 None admin] PATH_INFO: `/v3/projects/62daff6eeea54042a2879442094ef1b9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:50.418208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a23a1b-8650-471e-bd10-7389f72968f3 None admin] RBAC: Authorizing `identity:get_project(project_id=62daff6eeea54042a2879442094ef1b9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:50.419663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a23a1b-8650-471e-bd10-7389f72968f3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:50.421657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 226/452] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:07:50 2022] GET /identity/v3/projects/62daff6eeea54042a2879442094ef1b9 => generated 303 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:50.426888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a23a1b-8650-471e-bd10-7389f72968f3 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:50.500885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6abe6b-18fe-4708-af73-166f60139b46 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:50.502871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:50.528729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a23a1b-8650-471e-bd10-7389f72968f3 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:50.535899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:50.536516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:50.536797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:50.537112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:50.537940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] RBAC: Adding query filter params (group.id=c6c89047089442ed811ba216f2919811, role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=ee8a7cb4ed304a5598e10ca71a963013) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:50.538248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:50.539700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:50.545515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 227/453] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:50 2022] GET /identity/v3/role_assignments?group.id=c6c89047089442ed811ba216f2919811&scope.project.id=ee8a7cb4ed304a5598e10ca71a963013&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 613 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:50.562689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f203c4-952d-4b5c-8713-0b2a3adbf1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:50.563347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f203c4-952d-4b5c-8713-0b2a3adbf1a3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:50.563624 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f203c4-952d-4b5c-8713-0b2a3adbf1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:50.563893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f203c4-952d-4b5c-8713-0b2a3adbf1a3 None admin] PATH_INFO: `/v3/groups/897a344a4ae4412fb3c689cbce72ce41` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:50.565112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f203c4-952d-4b5c-8713-0b2a3adbf1a3 None admin] RBAC: Authorizing `identity:get_group(group_id=897a344a4ae4412fb3c689cbce72ce41)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:50.566624 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f203c4-952d-4b5c-8713-0b2a3adbf1a3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:50.568329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 227/454] 10.222.0.109 () {56 vars in 1185 bytes} [Thu Aug 4 22:07:50 2022] GET /identity/v3/groups/897a344a4ae4412fb3c689cbce72ce41 => generated 227 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:07:50.575000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:50.578932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:50.682567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5bc3f-b74c-4695-be52-7ce6530daaa1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:50.717995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b1d06-fdde-4fd8-9fa9-7712e0eea810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:50.718619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b1d06-fdde-4fd8-9fa9-7712e0eea810 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:50.718898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b1d06-fdde-4fd8-9fa9-7712e0eea810 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:50.719166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b1d06-fdde-4fd8-9fa9-7712e0eea810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:50.719949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b1d06-fdde-4fd8-9fa9-7712e0eea810 None admin] RBAC: Adding query filter params (group.id=897a344a4ae4412fb3c689cbce72ce41, role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=62daff6eeea54042a2879442094ef1b9) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:50.720254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b1d06-fdde-4fd8-9fa9-7712e0eea810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:50.721721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b1d06-fdde-4fd8-9fa9-7712e0eea810 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:50.728404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 228/455] 10.222.0.109 () {56 vars in 1381 bytes} [Thu Aug 4 22:07:50 2022] GET /identity/v3/role_assignments?group.id=897a344a4ae4412fb3c689cbce72ce41&scope.project.id=62daff6eeea54042a2879442094ef1b9&role.id=8f997477100248a9ace1cc28831caaac => generated 613 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:53.457824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3830dc1-ab1e-48d9-880b-6227558a4ce3 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:53.458320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3830dc1-ab1e-48d9-880b-6227558a4ce3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:53.458320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3830dc1-ab1e-48d9-880b-6227558a4ce3 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:53.460623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 228/456] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:53 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:53.466197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117220c-5bd6-44a4-b250-72c7d872086f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:53.467130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117220c-5bd6-44a4-b250-72c7d872086f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:53.467607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117220c-5bd6-44a4-b250-72c7d872086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:53.494396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-b117220c-5bd6-44a4-b250-72c7d872086f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:53.495971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b117220c-5bd6-44a4-b250-72c7d872086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:53.578394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:53.584971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 229/457] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:53.628320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3830dc1-ab1e-48d9-880b-6227558a4ce3 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:53.751952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bcce1-406f-4bd0-9947-dedac406b61d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:53.752247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bcce1-406f-4bd0-9947-dedac406b61d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:53.752530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bcce1-406f-4bd0-9947-dedac406b61d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:53.754204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 230/458] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:07:53.759866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e00e7-495f-4b96-8f0f-63190ac84f84 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:53.760148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e00e7-495f-4b96-8f0f-63190ac84f84 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:53.760422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e00e7-495f-4b96-8f0f-63190ac84f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:53.786158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3830dc1-ab1e-48d9-880b-6227558a4ce3 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:53.797446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e62e00e7-495f-4b96-8f0f-63190ac84f84 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:53.797446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e62e00e7-495f-4b96-8f0f-63190ac84f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:53.831578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e19c3d-d262-4f42-95b9-75c4f359485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:53.832234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e19c3d-d262-4f42-95b9-75c4f359485e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:53.832518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e19c3d-d262-4f42-95b9-75c4f359485e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:53.832819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e19c3d-d262-4f42-95b9-75c4f359485e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:53.845889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e19c3d-d262-4f42-95b9-75c4f359485e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:53.847693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e19c3d-d262-4f42-95b9-75c4f359485e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:53.849903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-18e19c3d-d262-4f42-95b9-75c4f359485e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:53.868231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:53.869140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:53.870006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:53.870006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:53.870006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:53.870006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:53.870006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 229/459] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:53.871058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b5a7-ffaf-41f8-b5ce-58c96b152f09 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:53.871342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b5a7-ffaf-41f8-b5ce-58c96b152f09 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:53.871614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b5a7-ffaf-41f8-b5ce-58c96b152f09 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:53.873516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 230/460] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:53.877019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 231/461] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:53.878218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b5a7-ffaf-41f8-b5ce-58c96b152f09 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:53.879034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c161-d382-433b-949b-68e242b23dec None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:53.879526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c161-d382-433b-949b-68e242b23dec None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:53.879989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c161-d382-433b-949b-68e242b23dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:53.900057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-1286c161-d382-433b-949b-68e242b23dec None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:53.901719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-1286c161-d382-433b-949b-68e242b23dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:53.978015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 232/462] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:53.979269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286c161-d382-433b-949b-68e242b23dec None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.017568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b5a7-ffaf-41f8-b5ce-58c96b152f09 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.053164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:54.053583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7befafd-5bd2-42e7-838c-67abfc682644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.054204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7befafd-5bd2-42e7-838c-67abfc682644 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.054488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7befafd-5bd2-42e7-838c-67abfc682644 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.054763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7befafd-5bd2-42e7-838c-67abfc682644 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.055454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7befafd-5bd2-42e7-838c-67abfc682644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:54.055834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7befafd-5bd2-42e7-838c-67abfc682644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.057317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7befafd-5bd2-42e7-838c-67abfc682644 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.072804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 231/463] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:54.074130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a40f4-8e72-47b2-90f3-50963291b416 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.074428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a40f4-8e72-47b2-90f3-50963291b416 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.074702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a40f4-8e72-47b2-90f3-50963291b416 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.076208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 232/464] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:54.076920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a40f4-8e72-47b2-90f3-50963291b416 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.104400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286c161-d382-433b-949b-68e242b23dec None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.141801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.142446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.142719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.142986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.149806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:54.154330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.155935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:54.170567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:54.171455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:54.172283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:54.172283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:54.172283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:54.172283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:54.172283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 233/465] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:54.172536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.202880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a40f4-8e72-47b2-90f3-50963291b416 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.239966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29b64c-e84d-48da-87f0-4dfa289fb704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.240709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29b64c-e84d-48da-87f0-4dfa289fb704 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.240989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29b64c-e84d-48da-87f0-4dfa289fb704 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.241258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29b64c-e84d-48da-87f0-4dfa289fb704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.252492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29b64c-e84d-48da-87f0-4dfa289fb704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.253980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29b64c-e84d-48da-87f0-4dfa289fb704 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-7c29b64c-e84d-48da-87f0-4dfa289fb704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:54.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:54.270067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:54.271977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:54.271977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:54.271977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:54.271977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:54.271977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 233/466] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:54.271977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1896c9-f4e0-4213-9d9d-2e9b3d696bd3 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.271977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1896c9-f4e0-4213-9d9d-2e9b3d696bd3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.272376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1896c9-f4e0-4213-9d9d-2e9b3d696bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.286713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abad01-4268-4c0f-a355-51088b2a5a47 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.290031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8c1896c9-f4e0-4213-9d9d-2e9b3d696bd3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:54.291200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8c1896c9-f4e0-4213-9d9d-2e9b3d696bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:54.298252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:54.325045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e149c6-709d-4ea7-b3f1-f68ddd809003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.325700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e149c6-709d-4ea7-b3f1-f68ddd809003 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.325983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e149c6-709d-4ea7-b3f1-f68ddd809003 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.326260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e149c6-709d-4ea7-b3f1-f68ddd809003 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.326937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e149c6-709d-4ea7-b3f1-f68ddd809003 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.328380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e149c6-709d-4ea7-b3f1-f68ddd809003 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.361438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 234/467] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:54 2022] POST /identity/v3/users => generated 247 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:54.361438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e149c6-709d-4ea7-b3f1-f68ddd809003 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.375709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 234/468] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:54.376785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1896c9-f4e0-4213-9d9d-2e9b3d696bd3 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.424306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:54.488133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e149c6-709d-4ea7-b3f1-f68ddd809003 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.502125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1896c9-f4e0-4213-9d9d-2e9b3d696bd3 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.535102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.536176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.536622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.537690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.538448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:54.538783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.540194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.542481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.543957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.544440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.544904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.545907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:54.546407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.548041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.559825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 235/469] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:54.560788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.563392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 235/470] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:54.564406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.596051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:54.682409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0a60f-2009-48b4-b14d-1dd4671226ee None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.688392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7278cbd-2ae0-410a-a87c-bd5730c72191 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.689966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:54.719268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.719919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.720207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.720506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.725506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd1db9-461f-495e-bc3f-d2b76b7b24a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.726119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd1db9-461f-495e-bc3f-d2b76b7b24a5 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.726395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd1db9-461f-495e-bc3f-d2b76b7b24a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.726666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd1db9-461f-495e-bc3f-d2b76b7b24a5 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.727342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd1db9-461f-495e-bc3f-d2b76b7b24a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.728793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd1db9-461f-495e-bc3f-d2b76b7b24a5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.731115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.732556 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:54.748227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:54.749182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:54.750051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:54.750051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:54.750051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:54.750051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:54.750051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 236/471] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:54.750319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.764526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 236/472] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:54 2022] POST /identity/v3/users => generated 249 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:54.764526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387461-ba13-47c7-a6e9-94fc1991651c None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.764526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387461-ba13-47c7-a6e9-94fc1991651c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.764526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387461-ba13-47c7-a6e9-94fc1991651c None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.764526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 237/473] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:54.764526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c387461-ba13-47c7-a6e9-94fc1991651c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.854114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:54.883637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b523ca3-3c06-41a7-9baa-537cc19f3bd3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.901057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c387461-ba13-47c7-a6e9-94fc1991651c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:54.925832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3608bba-223e-427a-9aa6-470c0052e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.926468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3608bba-223e-427a-9aa6-470c0052e060 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.926752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3608bba-223e-427a-9aa6-470c0052e060 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.927026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3608bba-223e-427a-9aa6-470c0052e060 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.928353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3608bba-223e-427a-9aa6-470c0052e060 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.929861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3608bba-223e-427a-9aa6-470c0052e060 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.951041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:54.951268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.952093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.952446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.953733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:54.953733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:54.955030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:54.955331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 237/474] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:54 2022] POST /identity/v3/users => generated 249 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:54.956654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0e8e3-69e8-405d-b163-162773e28621 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:54.956977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0e8e3-69e8-405d-b163-162773e28621 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:54.957229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0e8e3-69e8-405d-b163-162773e28621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:54.979748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 238/475] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:54.985464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-c2f0e8e3-69e8-405d-b163-162773e28621 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:54.986267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:54.986767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c2f0e8e3-69e8-405d-b163-162773e28621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:55.007253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:55.089353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 238/476] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:55.116395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b9748-f233-4322-a8da-b70b3949e0f8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:55.124125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0e8e3-69e8-405d-b163-162773e28621 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:55.159249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21b164-650a-4232-9168-d4dd812b114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:55.160036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21b164-650a-4232-9168-d4dd812b114e None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:55.160423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21b164-650a-4232-9168-d4dd812b114e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:55.160705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21b164-650a-4232-9168-d4dd812b114e None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:55.161395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21b164-650a-4232-9168-d4dd812b114e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:55.163042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21b164-650a-4232-9168-d4dd812b114e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:55.190399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 239/477] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:07:54 2022] POST /identity/v3/users => generated 250 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:07:55.205104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:55.259512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0e8e3-69e8-405d-b163-162773e28621 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:55.306809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:55.307461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:55.307744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:55.308020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:55.308770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:55.310222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:55.320597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.sql.core [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 22:07:55.320597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 22:07:55.320597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [parameters: {'id': 'fb6a7a2253774bac97d9c5c529cab6d5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 22:07:55.320597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91702) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application conn.query(q) Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application result.read() Aug 04 22:07:55.338701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 22:07:55.340767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application next(self.gen) Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application yield resource Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application next(self.gen) Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise self.value Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application session.commit() Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 22:07:55.343003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self.session.flush() Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application compat.raise_( Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 22:07:55.346919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 22:07:55.348991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application conn.query(q) Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application result.read() Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 22:07:55.353486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [parameters: {'id': 'fb6a7a2253774bac97d9c5c529cab6d5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:55.357498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 22:07:55.359501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:55.361467 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 239/478] 10.222.0.109 () {60 vars in 1136 bytes} [Thu Aug 4 22:07:55 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:55.361609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21b164-650a-4232-9168-d4dd812b114e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:55.437029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:55.477370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21b164-650a-4232-9168-d4dd812b114e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:55.516130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001596c6-d8e9-442e-8e94-d145b8c950f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:55.516863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001596c6-d8e9-442e-8e94-d145b8c950f7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:55.517060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001596c6-d8e9-442e-8e94-d145b8c950f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:55.517337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001596c6-d8e9-442e-8e94-d145b8c950f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:55.531789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001596c6-d8e9-442e-8e94-d145b8c950f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-001596c6-d8e9-442e-8e94-d145b8c950f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:55.537667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:55.538534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:55.539352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 240/479] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:07:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:07:55.546374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:55.665981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928569-9686-423c-a347-fdb12c694ff9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:55.694556 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:55.708276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398210a4-be67-4056-9b25-454708be2d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:55.710387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398210a4-be67-4056-9b25-454708be2d83 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:55.710862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398210a4-be67-4056-9b25-454708be2d83 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:55.711275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398210a4-be67-4056-9b25-454708be2d83 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:55.712161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398210a4-be67-4056-9b25-454708be2d83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:55.712605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398210a4-be67-4056-9b25-454708be2d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:55.714167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398210a4-be67-4056-9b25-454708be2d83 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:55.723280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 240/480] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:55 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:56.788425 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5548-71a7-4e14-b182-8350ac34793d None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:56.789523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5548-71a7-4e14-b182-8350ac34793d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:56.790012 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5548-71a7-4e14-b182-8350ac34793d None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:56.791929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 241/481] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:56 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:56.797106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dc30b-fbb0-48dd-8a33-cf401a516440 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:56.797599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dc30b-fbb0-48dd-8a33-cf401a516440 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:56.797687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dc30b-fbb0-48dd-8a33-cf401a516440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:56.822463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-928dc30b-fbb0-48dd-8a33-cf401a516440 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:56.824014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-928dc30b-fbb0-48dd-8a33-cf401a516440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:56.918115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 241/482] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:56.953498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e5548-71a7-4e14-b182-8350ac34793d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:57.080429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e5548-71a7-4e14-b182-8350ac34793d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:57.109399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:57.119123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ff2b-6e43-4da4-a980-044ecb09498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:57.119773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ff2b-6e43-4da4-a980-044ecb09498a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.120054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ff2b-6e43-4da4-a980-044ecb09498a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.120332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ff2b-6e43-4da4-a980-044ecb09498a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.132416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ff2b-6e43-4da4-a980-044ecb09498a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a9b2ff2b-6e43-4da4-a980-044ecb09498a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:57.137429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:57.138304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:57.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 242/483] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:07:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:57.144868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dc30b-fbb0-48dd-8a33-cf401a516440 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:57.291970 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dc30b-fbb0-48dd-8a33-cf401a516440 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:57.320414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:57.341171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c70e9-200b-4a37-aee8-665136d16329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:57.341753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c70e9-200b-4a37-aee8-665136d16329 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.342027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c70e9-200b-4a37-aee8-665136d16329 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.342294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c70e9-200b-4a37-aee8-665136d16329 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.343001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c70e9-200b-4a37-aee8-665136d16329 None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:57.343298 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c70e9-200b-4a37-aee8-665136d16329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:57.344660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c70e9-200b-4a37-aee8-665136d16329 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:57.357803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 242/484] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:07:57 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:57.362847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ff2b-6e43-4da4-a980-044ecb09498a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:57.403577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b77b4-e587-456e-b86f-747fcf9449a6 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.403895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b77b4-e587-456e-b86f-747fcf9449a6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.404118 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b77b4-e587-456e-b86f-747fcf9449a6 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.405737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 243/485] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:57.410644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee44ff-175d-459c-b3e5-1d00e6198deb None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.410924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee44ff-175d-459c-b3e5-1d00e6198deb None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.411193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee44ff-175d-459c-b3e5-1d00e6198deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.450487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-48ee44ff-175d-459c-b3e5-1d00e6198deb None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:57.451689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-48ee44ff-175d-459c-b3e5-1d00e6198deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:57.505216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ff2b-6e43-4da4-a980-044ecb09498a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:57.567477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:57.568081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 244/486] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:07:57.569251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.569581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.569860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.589198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:57.590734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:57.592301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee44ff-175d-459c-b3e5-1d00e6198deb None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:57.611919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:57.612835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:57.614133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:57.614133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:57.614133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:57.614133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:57.617100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 243/487] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:57.623647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:57.688318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:57.743159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee44ff-175d-459c-b3e5-1d00e6198deb None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:57.749333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8618e-8062-4b51-bf6d-b9f12cd076b9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:57.757865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:57.783008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf05b72-23ef-426c-bffa-f3ec2a4e0e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:57.783656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf05b72-23ef-426c-bffa-f3ec2a4e0e33 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.783931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf05b72-23ef-426c-bffa-f3ec2a4e0e33 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.784197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf05b72-23ef-426c-bffa-f3ec2a4e0e33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.787219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c8ca8-6217-4009-84c5-1e5c3fe7ccdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:57.788090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c8ca8-6217-4009-84c5-1e5c3fe7ccdf None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.788562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c8ca8-6217-4009-84c5-1e5c3fe7ccdf None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.789600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c8ca8-6217-4009-84c5-1e5c3fe7ccdf None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.790488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c8ca8-6217-4009-84c5-1e5c3fe7ccdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:57.790983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c8ca8-6217-4009-84c5-1e5c3fe7ccdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:57.792547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c8ca8-6217-4009-84c5-1e5c3fe7ccdf None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:57.796095 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf05b72-23ef-426c-bffa-f3ec2a4e0e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-ecf05b72-23ef-426c-bffa-f3ec2a4e0e33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:57.801999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:57.809621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 245/488] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:07:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:57.810707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da368e70-8863-4af3-8e54-087cdcd99b8d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.811412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da368e70-8863-4af3-8e54-087cdcd99b8d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.811412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da368e70-8863-4af3-8e54-087cdcd99b8d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.813873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 246/489] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:57 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:57.814849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da368e70-8863-4af3-8e54-087cdcd99b8d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:57.823703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 244/490] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:57.825320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f508d3f-58f3-48ff-8577-0a6a9f232ee4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:57.825928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f508d3f-58f3-48ff-8577-0a6a9f232ee4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:57.826397 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f508d3f-58f3-48ff-8577-0a6a9f232ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:57.848554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0f508d3f-58f3-48ff-8577-0a6a9f232ee4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:57.850253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0f508d3f-58f3-48ff-8577-0a6a9f232ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:57.928813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 245/491] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:57.930188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f508d3f-58f3-48ff-8577-0a6a9f232ee4 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:57.976043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da368e70-8863-4af3-8e54-087cdcd99b8d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:57.981332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:58.016623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.017246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.017567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.017837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.018545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:58.019097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:58.020968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:58.029699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 247/492] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:07:57 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:58.034047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.069501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:58.070619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f508d3f-58f3-48ff-8577-0a6a9f232ee4 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.106322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc0c6e-a8a9-458e-b4f6-1b914ac40cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.106892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0c6e-a8a9-458e-b4f6-1b914ac40cec None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.107167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0c6e-a8a9-458e-b4f6-1b914ac40cec None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.107437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0c6e-a8a9-458e-b4f6-1b914ac40cec None admin] PATH_INFO: `/v3/users/nova` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.117339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc0c6e-a8a9-458e-b4f6-1b914ac40cec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:58.118877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc0c6e-a8a9-458e-b4f6-1b914ac40cec None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-75cc0c6e-a8a9-458e-b4f6-1b914ac40cec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:07:58.136794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:58.137759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:07:58.138649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:58.144662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 246/493] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:07:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:58.144662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4b051-335b-4dcb-a94a-f6a966477f59 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.144662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4b051-335b-4dcb-a94a-f6a966477f59 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.144662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4b051-335b-4dcb-a94a-f6a966477f59 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.144662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 247/494] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:58 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:58.145143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4b051-335b-4dcb-a94a-f6a966477f59 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.148908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f4aa0-5685-4de4-83b5-eaea41153aee None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.183726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587d2a8-29bb-4fef-b778-2378b1e6e73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.184367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d2a8-29bb-4fef-b778-2378b1e6e73d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.184641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d2a8-29bb-4fef-b778-2378b1e6e73d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.184908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d2a8-29bb-4fef-b778-2378b1e6e73d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.196065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587d2a8-29bb-4fef-b778-2378b1e6e73d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-f587d2a8-29bb-4fef-b778-2378b1e6e73d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:58.201342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:58.202265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:58.203063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 248/495] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:58.203720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587d2a8-29bb-4fef-b778-2378b1e6e73d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.253162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:58.265194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4b051-335b-4dcb-a94a-f6a966477f59 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.302572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa312957-d977-497e-adb5-f242b2fc4cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.303211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa312957-d977-497e-adb5-f242b2fc4cc3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.303496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa312957-d977-497e-adb5-f242b2fc4cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.303763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa312957-d977-497e-adb5-f242b2fc4cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.304636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:58.311649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587d2a8-29bb-4fef-b778-2378b1e6e73d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.314830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa312957-d977-497e-adb5-f242b2fc4cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:58.316306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa312957-d977-497e-adb5-f242b2fc4cc3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-fa312957-d977-497e-adb5-f242b2fc4cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:58.331091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:58.331994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:58.338304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:58.338304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:58.338304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:58.338304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:58.338304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 248/496] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:58.338304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75654f5-0d17-4b8f-affc-54d56a750804 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.338304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75654f5-0d17-4b8f-affc-54d56a750804 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.338304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75654f5-0d17-4b8f-affc-54d56a750804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.352570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e75654f5-0d17-4b8f-affc-54d56a750804 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:58.353811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e75654f5-0d17-4b8f-affc-54d56a750804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:58.357077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.357666 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.357946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.358211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.358994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:58.359295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:58.360702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:58.403690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 249/497] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:58.404401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.435265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 249/498] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:58.436456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75654f5-0d17-4b8f-affc-54d56a750804 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.512328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:58.547288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e38e3c-4ef4-4847-997c-4a2895ff3092 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.550990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75654f5-0d17-4b8f-affc-54d56a750804 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.581772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.582405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.582689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.582970 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.584494 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:58.584815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:58.586288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:58.593762 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.594390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.594669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.599350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.599533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 250/499] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:58.599657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:58.599773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.599890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:58.599890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:58.611371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 250/500] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:58.612142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.615284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:58.705018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0975bb-457c-4224-a6ea-f89d77406f2e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.742190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa8c0c-38b7-426d-ae5a-6141364ca8d8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.742872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f04216-e41a-414f-a711-e6360ddbd5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.743501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04216-e41a-414f-a711-e6360ddbd5d6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.743787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04216-e41a-414f-a711-e6360ddbd5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.744062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04216-e41a-414f-a711-e6360ddbd5d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.772663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:58.778290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f0c1b-1687-4270-8a3c-a5bbdeb044bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.778930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f0c1b-1687-4270-8a3c-a5bbdeb044bd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.779203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f0c1b-1687-4270-8a3c-a5bbdeb044bd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.779472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f0c1b-1687-4270-8a3c-a5bbdeb044bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.790247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f0c1b-1687-4270-8a3c-a5bbdeb044bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-175f0c1b-1687-4270-8a3c-a5bbdeb044bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:07:58.795610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:58.796537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:58.797332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 251/501] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:58.798039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d728-59ef-4936-aa75-64da659083d2 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.798990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d728-59ef-4936-aa75-64da659083d2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.798990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d728-59ef-4936-aa75-64da659083d2 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.799668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 252/502] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:07:58 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:07:58.801652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d728-59ef-4936-aa75-64da659083d2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.817518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f04216-e41a-414f-a711-e6360ddbd5d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:58.819010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f04216-e41a-414f-a711-e6360ddbd5d6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:58.834066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:58.903878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d728-59ef-4936-aa75-64da659083d2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:58.917768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 251/503] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:58.918940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f04216-e41a-414f-a711-e6360ddbd5d6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:58.937550 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:58.938194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:58.938475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:58.938750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:58.950014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:58.951479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:58.965630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:58.966657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:58.967874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:58.967874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:58.967874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:58.967874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:58.967874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 253/504] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:58.967874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.014549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.023211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f04216-e41a-414f-a711-e6360ddbd5d6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.038256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.058582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00563707-3cc7-470e-8f1a-d2a5a7ae7534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.059222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00563707-3cc7-470e-8f1a-d2a5a7ae7534 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.059504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00563707-3cc7-470e-8f1a-d2a5a7ae7534 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.059776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00563707-3cc7-470e-8f1a-d2a5a7ae7534 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.066642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00563707-3cc7-470e-8f1a-d2a5a7ae7534 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.068086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bf8d4-1df2-4f83-9587-caabaff1c1a7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.068207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00563707-3cc7-470e-8f1a-d2a5a7ae7534 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-00563707-3cc7-470e-8f1a-d2a5a7ae7534 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:07:59.080802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:59.081891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:59.083161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 252/505] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:59.083947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff1c31-55f1-4f0e-a2f7-580c4b0461bc None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.083947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff1c31-55f1-4f0e-a2f7-580c4b0461bc None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.084063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff1c31-55f1-4f0e-a2f7-580c4b0461bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.102177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2bff1c31-55f1-4f0e-a2f7-580c4b0461bc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:07:59.103487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2bff1c31-55f1-4f0e-a2f7-580c4b0461bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:07:59.104406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.105092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.105398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.105717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.108207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:59.108207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.108207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.116871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 254/506] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:07:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:59.117510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.181869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 253/507] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:07:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:07:59.182629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff1c31-55f1-4f0e-a2f7-580c4b0461bc None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.222949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1937fe-738f-47f6-81a8-48e48415ebeb None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.231553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.259949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.260592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.260870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.261142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.271216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.272779 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:59.287930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:59.288994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:59.290092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:59.290092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:59.290092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:59.290092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:59.290092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 255/508] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:59.290430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.297900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff1c31-55f1-4f0e-a2f7-580c4b0461bc None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.300351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.334306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.334925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.335223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.335476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.336167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:59.336469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.337881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.349029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 254/509] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:59.350576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.388162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935fc17-97a2-48f3-a14c-637d7236ff34 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.422473 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.423102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.423382 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.423647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.424429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:59.424729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.426189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.449723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a576b-39f4-4720-86a9-a286bc0a0cac None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.461581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 256/510] 10.222.0.109 () {56 vars in 1142 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:59.462401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.481608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.485894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.486653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.486939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.487215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.497860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.499294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:59.513352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:59.514388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:59.515258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:59.515258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:59.515258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:59.515258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:07:59.515258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 255/511] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:59.515258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.535608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.563398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca53098-5b20-43cc-a548-1ab694a47065 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.596529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.597246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.597564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.597840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.608707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.610191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.617913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75922e-d589-4950-87d3-c6e33080c5b6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:07:59.626070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:59.627077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:59.633038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:07:59.633038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:07:59.633038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:07:59.633038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:59.633038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 257/512] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:59.633038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.653772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.654399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.654678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.654951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.655636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:59.655936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.657300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.669650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 256/513] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:59.670441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.695528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.729009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebf986-e3d0-41ba-a265-bdb6061156c6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.746911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.763629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.764268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.764546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.765040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.770630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860848e-03f0-4e05-9f8f-a7273de23857 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.773859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.775329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:07:59.786488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:07:59.787530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:07:59.788535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 258/514] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:07:59.789093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.810330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065283eb-cefc-491f-929a-10df71c8c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.810951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065283eb-cefc-491f-929a-10df71c8c633 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.811233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065283eb-cefc-491f-929a-10df71c8c633 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.811508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065283eb-cefc-491f-929a-10df71c8c633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.879633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065283eb-cefc-491f-929a-10df71c8c633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.881636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065283eb-cefc-491f-929a-10df71c8c633 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.886285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82702b4b-e532-4bce-b20b-6c77ff41b9fa None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:07:59.912236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.921890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:07:59.922494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:07:59.922776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:07:59.923045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:07:59.923783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:07:59.924086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:07:59.925525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:07:59.936673 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 259/515] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:07:59.937479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:07:59.962159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:07:59.985654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 257/516] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:07:59.986408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065283eb-cefc-491f-929a-10df71c8c633 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.036137 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e5d6f-4d60-417f-a2dc-7763f5bb1517 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.072155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.072779 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.073057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.073329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.074081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:00.074381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.075757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.087566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065283eb-cefc-491f-929a-10df71c8c633 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.088121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 260/517] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:00.088772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.127670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.128320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.128600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.128873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.136285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.137777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.140231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:00.150326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:00.151288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:00.152117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 258/518] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:00.152583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.186032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:00.195117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442023ec-3778-49f9-a5aa-bf8cfa340b21 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.227743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.228360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.228639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.228914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.229736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:00.230043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.231499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.248782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 261/519] 10.222.0.109 () {56 vars in 1142 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:00.249546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.252154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ea0e5-07d4-4ef1-84ff-619b56a08f27 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.286591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.287253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.287532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.287802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.294674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.296117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:00.309263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:00.310177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:00.311057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 259/520] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:00.311598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.355317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494828c-9f39-4c6c-bc0d-d9886023f0b0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.362385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:00.395124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.395854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.396147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.396456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.407342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:00.407469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.408828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.414783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3c056-d3e8-432e-ab01-1693454d859d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:00.423859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:00.424906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:00.425989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:00.425989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:00.425989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:00.425989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:00.425989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 262/521] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:00.425989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.455008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a4685-351a-4c38-b5d1-ccbdcbabc9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.455616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4685-351a-4c38-b5d1-ccbdcbabc9cb None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.455889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4685-351a-4c38-b5d1-ccbdcbabc9cb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.456393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4685-351a-4c38-b5d1-ccbdcbabc9cb None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.456817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a4685-351a-4c38-b5d1-ccbdcbabc9cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.458285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a4685-351a-4c38-b5d1-ccbdcbabc9cb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.479361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 260/522] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:08:00 2022] POST /identity/v3/users => generated 248 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:00.480254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a4685-351a-4c38-b5d1-ccbdcbabc9cb None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.534411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19a13-4590-4b3e-ad65-1491a0a93e52 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.575626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.576256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.576538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.576818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.577614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:00.577925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.579654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.595148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a4685-351a-4c38-b5d1-ccbdcbabc9cb None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.607228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 263/523] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:00.607228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.627844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:00.643139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952cd48-6a0d-441b-946c-c95823cc8c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.644044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952cd48-6a0d-441b-946c-c95823cc8c23 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.644958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952cd48-6a0d-441b-946c-c95823cc8c23 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.644958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952cd48-6a0d-441b-946c-c95823cc8c23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.675049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:00.719296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752dfe5-1487-426f-a225-3f395ac7aa67 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.727125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952cd48-6a0d-441b-946c-c95823cc8c23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.728712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952cd48-6a0d-441b-946c-c95823cc8c23 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.752994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122522b-02da-4180-988e-840f41f1048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.753642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122522b-02da-4180-988e-840f41f1048e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.753916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122522b-02da-4180-988e-840f41f1048e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.754185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122522b-02da-4180-988e-840f41f1048e None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.754967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122522b-02da-4180-988e-840f41f1048e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:00.755851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122522b-02da-4180-988e-840f41f1048e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.757520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122522b-02da-4180-988e-840f41f1048e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.775153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 264/524] 10.222.0.109 () {56 vars in 1144 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:00.775977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122522b-02da-4180-988e-840f41f1048e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.836319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 261/525] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:08:00.837483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952cd48-6a0d-441b-946c-c95823cc8c23 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.858412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:00.882172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122522b-02da-4180-988e-840f41f1048e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.897845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:00.919545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.920202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.920439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.920707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.921545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:00.921845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.923334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:00.940321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 265/526] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:00.940321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:00.951322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952cd48-6a0d-441b-946c-c95823cc8c23 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:00.990054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:00.990677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:00.990984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:00.991220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:00.992010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=41d9317e24714de1a310198bb35eb00b) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:00.992313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:00.993762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.009773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 262/527] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/role_assignments?user.id=41d9317e24714de1a310198bb35eb00b&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:01.010601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.053883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062d369-bbe2-417f-bd1a-da820155c613 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.093029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce34f3-8f21-4ef1-a210-f3c8aab4c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.093757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce34f3-8f21-4ef1-a210-f3c8aab4c932 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.094175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce34f3-8f21-4ef1-a210-f3c8aab4c932 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.094537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce34f3-8f21-4ef1-a210-f3c8aab4c932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.098726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:01.134255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55201013-fdcf-41e9-9f6b-bade5cefaed4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.158851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:01.186378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.187025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.187307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.187571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.195407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce34f3-8f21-4ef1-a210-f3c8aab4c932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.197761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce34f3-8f21-4ef1-a210-f3c8aab4c932 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.202851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.205352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:01.222112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:01.223023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:01.224200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:01.224200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:01.224200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:01.224200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:01.228282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 263/528] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:01.229232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.320254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 266/529] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:08:01.321130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce34f3-8f21-4ef1-a210-f3c8aab4c932 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.343161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58781af4-502c-446f-bfd5-b0ee9a3e2a20 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.362519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:01.384227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.385318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.385824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.386346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.394211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.396050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:01.410417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:01.411293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:01.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:01.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:01.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:01.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:01.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:01.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:01.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:01.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:01.419768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 264/530] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:01.420948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.435459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce34f3-8f21-4ef1-a210-f3c8aab4c932 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.454642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:01.472520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.473141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.473437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.473725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.474936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:01.475243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.476606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.489225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 267/531] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:01.490748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.545610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db296659-aee7-41ec-8b59-c1b5075c5e81 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.581160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.582014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.582126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.582358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.594314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.595682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:01.611037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:01.611928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:01.612994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:01.612994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:01.612994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:01.612994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:01.612994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 265/532] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:01.613347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.629673 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938be33-46a4-45e9-bcf9-a963b75ead8a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.653835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:01.677974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.679093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.679486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.681454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.682551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:01.683273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.685102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.701615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 268/533] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:01.705674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.711558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:01.732734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f5e7-b321-4d4c-89c1-7d31e02eb5d2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.768105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.769117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.769299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.769583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.777353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.778846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:01.791373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:01.792235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:01.793063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:01.793063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:01.793063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:01.793063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:01.793063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:01.793063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:01.793063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:01.793063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:01.797213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 266/534] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:01.798094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.842873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635f411-b3d8-4f60-93d7-f23b786d249e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.880656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.881590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.882096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.882569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.883454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:01.883973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.885600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.898481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 269/535] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:01.899647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.907794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0320de7-dd6a-491d-8822-dd88ad637c55 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:01.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:01.943959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:01.944633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:01.944910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:01.945180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:01.946147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=1d0422cd82114045a9fff7df29bc3906) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:01.946353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:01.947891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:01.961822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 267/536] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/role_assignments?user.id=1d0422cd82114045a9fff7df29bc3906&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:01.962838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:01.966725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:02.017632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf8ce6-1a5c-4be2-b821-5862def8dd32 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:02.057168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:02.058243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:02.058726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:02.059202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:02.060183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:02.060699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:02.062450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:02.077697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 270/537] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:02.085659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:02.201382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9f970-6d1c-4029-964e-4a2d8dd8b6f3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:02.220445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:02.266190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3008-e7d5-44e5-8ae6-673527096b1c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:02.267137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:02.268092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:02.269023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:02.271375 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:02.284915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:02.289576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:02.312897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:02.313916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:02.314741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:02.314741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:02.314741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:02.314741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:02.314741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:02.314741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:02.314741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:02.314741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:02.320148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 268/538] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:02.327256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:02.338985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:02.339618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:02.339897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:02.340173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:02.340968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=33183e28de8f487f82f92f0151e7c73c) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:02.341283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:02.344789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:02.349785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 271/539] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:02 2022] GET /identity/v3/role_assignments?user.id=33183e28de8f487f82f92f0151e7c73c&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:02.462797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:02.524535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b8762-fed6-4788-ad2e-be2529a55ca2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:02.570170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97caf82-6c81-41ed-bb38-cc5858592393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:02.570821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97caf82-6c81-41ed-bb38-cc5858592393 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:02.571100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97caf82-6c81-41ed-bb38-cc5858592393 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:02.571376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97caf82-6c81-41ed-bb38-cc5858592393 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:02.572145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97caf82-6c81-41ed-bb38-cc5858592393 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:02.572450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97caf82-6c81-41ed-bb38-cc5858592393 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:02.574023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97caf82-6c81-41ed-bb38-cc5858592393 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:02.597987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 269/540] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:02.604416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:02.737317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0cd0-f29c-4a84-a129-7fcd9704fb6d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:02.737532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0cd0-f29c-4a84-a129-7fcd9704fb6d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:02.737660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854df5d-962e-4c86-8ddd-cc216d33629f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:02.737808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0cd0-f29c-4a84-a129-7fcd9704fb6d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:02.739438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 270/541] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:02.745037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1f928-e005-49e4-8eb4-c7f8acd0b9b6 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:02.745319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1f928-e005-49e4-8eb4-c7f8acd0b9b6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:02.745618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1f928-e005-49e4-8eb4-c7f8acd0b9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:02.780733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7ee1f928-e005-49e4-8eb4-c7f8acd0b9b6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:02.781977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7ee1f928-e005-49e4-8eb4-c7f8acd0b9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:02.799106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:02.799735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:02.800017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:02.800295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:02.801086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=5455c117e349480ba8b8c020b767bec5) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:02.801396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:02.806250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:02.826498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 272/542] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:02 2022] GET /identity/v3/role_assignments?user.id=5455c117e349480ba8b8c020b767bec5&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:02.895692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 271/543] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:08:02.934392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:02.938644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:03.078900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018255-0cde-4658-bc28-598104aa0b36 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:03.117218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8aa66-04c0-4937-b096-6861c976eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:03.117921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8aa66-04c0-4937-b096-6861c976eac4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:03.118196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8aa66-04c0-4937-b096-6861c976eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:03.118463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8aa66-04c0-4937-b096-6861c976eac4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:03.128624 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8aa66-04c0-4937-b096-6861c976eac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a7a8aa66-04c0-4937-b096-6861c976eac4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:03.134966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:03.135843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:03.136652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 273/544] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:03.142201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1f928-e005-49e4-8eb4-c7f8acd0b9b6 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:03.208342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45531514-e07b-4f12-ab83-7725f5c164a8 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:03.208698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45531514-e07b-4f12-ab83-7725f5c164a8 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:03.208886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45531514-e07b-4f12-ab83-7725f5c164a8 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:03.210395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 274/545] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:03.215244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0852ff-d486-4da3-846e-aeef057987bd None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:03.215788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0852ff-d486-4da3-846e-aeef057987bd None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:03.216251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0852ff-d486-4da3-846e-aeef057987bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:03.228504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:03.235098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-4a0852ff-d486-4da3-846e-aeef057987bd None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:03.236546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4a0852ff-d486-4da3-846e-aeef057987bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:03.307812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:03.308387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 275/546] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:08:03.332102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0852ff-d486-4da3-846e-aeef057987bd None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:03.339387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1f928-e005-49e4-8eb4-c7f8acd0b9b6 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:03.432323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:03.432974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:03.433258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:03.433563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:03.434295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:03.434595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:03.435988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:03.449394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 272/547] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:03 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:03.451635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0852ff-d486-4da3-846e-aeef057987bd None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:03.454777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:03.509780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a9ebd-bc9e-46bf-a7a2-ea443d9c6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:03.511664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9ebd-bc9e-46bf-a7a2-ea443d9c6164 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:03.512954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9ebd-bc9e-46bf-a7a2-ea443d9c6164 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:03.513431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9ebd-bc9e-46bf-a7a2-ea443d9c6164 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:03.531406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a9ebd-bc9e-46bf-a7a2-ea443d9c6164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-9d4a9ebd-bc9e-46bf-a7a2-ea443d9c6164 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:03.539560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:03.540442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:03.541262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 276/548] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:03.548467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a9ebd-bc9e-46bf-a7a2-ea443d9c6164 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:03.633007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f022f-0503-441e-8729-a38a8f843711 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:03.644392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:03.662456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:03.698072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:03.698724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:03.699003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:03.699276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:03.720054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:03.721541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:03.729699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a9ebd-bc9e-46bf-a7a2-ea443d9c6164 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:03.751596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:03.752484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:03.754063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:03.754063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:03.754063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:03.754063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:03.755581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 273/549] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:03.762362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:03.784950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:03.785715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:03.785993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:03.786873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:03.787872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:03.788174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:03.789596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:03.798337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 277/550] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:03 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:03.808551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:03.959631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0abf4-ca84-4a4f-a3c8-9d1977191838 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:03.975380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d661d-ad77-4c50-9a3d-359723a044fd None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:04.000098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:04.016413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.017057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.017420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.017709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.037189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:04.039223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:04.042142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.042771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.043051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.043326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.044019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:04.044325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:04.045703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:04.064616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 274/551] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:04.074423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:04.076493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:04.078813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:04.078813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:04.078813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:04.078813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:04.078813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 278/552] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:04.079458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:04.083824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:04.113503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:04.194272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9082ebd-df66-4a09-b976-16f22916b394 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:04.232929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.233543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.233827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.234100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.243863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:04.245434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:04.259289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:04.260313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:04.261245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 275/553] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:04.268253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:04.338106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e08c40-4b88-491e-b445-dfaab40c379c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:04.419313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efd025-022a-4d6b-91c3-eb982f88e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.420849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd025-022a-4d6b-91c3-eb982f88e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.421343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd025-022a-4d6b-91c3-eb982f88e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.421867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd025-022a-4d6b-91c3-eb982f88e1b0 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.423048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efd025-022a-4d6b-91c3-eb982f88e1b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:04.423567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efd025-022a-4d6b-91c3-eb982f88e1b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:04.425348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efd025-022a-4d6b-91c3-eb982f88e1b0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:04.442302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 279/554] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:04.444598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5035b9-fd12-4378-80cd-bd2c66bbb2be None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.445136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5035b9-fd12-4378-80cd-bd2c66bbb2be None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.445660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5035b9-fd12-4378-80cd-bd2c66bbb2be None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.452095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 280/555] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:04 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:04.452292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:04.452378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5035b9-fd12-4378-80cd-bd2c66bbb2be None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:04.473007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda2a5c-01df-491d-9599-b19f02b28af9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:04.482864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:04.514896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c9304-8e02-4771-9914-d9387928547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.515860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9304-8e02-4771-9914-d9387928547d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.516360 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9304-8e02-4771-9914-d9387928547d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.516861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9304-8e02-4771-9914-d9387928547d None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.517886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c9304-8e02-4771-9914-d9387928547d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:04.518641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c9304-8e02-4771-9914-d9387928547d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:04.520378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c9304-8e02-4771-9914-d9387928547d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:04.539428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 276/556] 10.222.0.109 () {56 vars in 1140 bytes} [Thu Aug 4 22:08:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:04.540966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c166f3b-38c2-4ec6-bb06-a8ddffb80076 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.541589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c166f3b-38c2-4ec6-bb06-a8ddffb80076 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.542071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c166f3b-38c2-4ec6-bb06-a8ddffb80076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.560627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5c166f3b-38c2-4ec6-bb06-a8ddffb80076 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:04.562167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5c166f3b-38c2-4ec6-bb06-a8ddffb80076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:04.566804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5035b9-fd12-4378-80cd-bd2c66bbb2be None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:04.603680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.604318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.604592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.604861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.612433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:04.613935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:04.627510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:04.628980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 281/557] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:08:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:04.630593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:04.649480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 277/558] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:04.650706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c166f3b-38c2-4ec6-bb06-a8ddffb80076 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:04.723547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:04.737440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f32a54-5c56-42d1-862e-cde7ac2b26e5 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:04.748896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:04.774261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7da51-ec1a-462e-a577-5aa309768111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.774757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7da51-ec1a-462e-a577-5aa309768111 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.775032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7da51-ec1a-462e-a577-5aa309768111 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.775300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7da51-ec1a-462e-a577-5aa309768111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.779200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c166f3b-38c2-4ec6-bb06-a8ddffb80076 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:04.816853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.818532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.819024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.819501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.820513 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:04.821022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:04.822700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:04.841686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 278/559] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:08:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:04.842828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:04.847175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7da51-ec1a-462e-a577-5aa309768111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:04.848700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7da51-ec1a-462e-a577-5aa309768111 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:04.939291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:04.954464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e889af4-52e3-4c23-a974-da2008bf87d6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:04.959851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 282/560] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:04 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:08:04.960770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7da51-ec1a-462e-a577-5aa309768111 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:04.993282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1bc22-ea83-428d-8a8d-b513510ccaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:04.994359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1bc22-ea83-428d-8a8d-b513510ccaa2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:04.994856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1bc22-ea83-428d-8a8d-b513510ccaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:04.995334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1bc22-ea83-428d-8a8d-b513510ccaa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:04.997382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:05.008129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1bc22-ea83-428d-8a8d-b513510ccaa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e7c1bc22-ea83-428d-8a8d-b513510ccaa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:05.014447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:05.015372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:05.016245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:05.024564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 279/561] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:05.025719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1bc22-ea83-428d-8a8d-b513510ccaa2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.062783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7da51-ec1a-462e-a577-5aa309768111 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.098986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.099932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.100314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.100691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.111866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:05.113431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:05.128574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:05.129491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:05.130895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:05.130895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:05.130895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:05.130895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:05.134845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 283/562] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:05.135801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.138418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1bc22-ea83-428d-8a8d-b513510ccaa2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.174832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdcd5a-edab-4566-bb9c-dcf7fee21ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.175474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdcd5a-edab-4566-bb9c-dcf7fee21ab5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.175753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdcd5a-edab-4566-bb9c-dcf7fee21ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.176024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdcd5a-edab-4566-bb9c-dcf7fee21ab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.181724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:05.187272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdcd5a-edab-4566-bb9c-dcf7fee21ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:05.188723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdcd5a-edab-4566-bb9c-dcf7fee21ab5 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e9fdcd5a-edab-4566-bb9c-dcf7fee21ab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:05.206091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:05.207451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:05.208912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:05.208912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:05.208912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:05.208912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:05.208912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 280/563] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:05.209383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60d3d2-c0a8-4cd4-9262-916ff8de84a8 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.209847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60d3d2-c0a8-4cd4-9262-916ff8de84a8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.210133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60d3d2-c0a8-4cd4-9262-916ff8de84a8 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.211768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 281/564] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:05.212913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60d3d2-c0a8-4cd4-9262-916ff8de84a8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.251845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000685f-4ad9-4a38-9e85-e4e5cb5d06b1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.261233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:05.286538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777b68b-cf2e-422b-8bd8-96071f386f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.287057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b68b-cf2e-422b-8bd8-96071f386f19 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.287340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b68b-cf2e-422b-8bd8-96071f386f19 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.287612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b68b-cf2e-422b-8bd8-96071f386f19 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.288324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777b68b-cf2e-422b-8bd8-96071f386f19 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:05.288629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777b68b-cf2e-422b-8bd8-96071f386f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:05.290057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777b68b-cf2e-422b-8bd8-96071f386f19 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:05.298069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 284/565] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:05.299541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661bff-2273-43b8-bbeb-8a579f0338da None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.299830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661bff-2273-43b8-bbeb-8a579f0338da None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.300107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661bff-2273-43b8-bbeb-8a579f0338da None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.301386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 285/566] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:05.302710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46661bff-2273-43b8-bbeb-8a579f0338da None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.322242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60d3d2-c0a8-4cd4-9262-916ff8de84a8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.355349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8c3d-36d5-46fa-ace1-119a7a953adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.355961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8c3d-36d5-46fa-ace1-119a7a953adc None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.356234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8c3d-36d5-46fa-ace1-119a7a953adc None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.356499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8c3d-36d5-46fa-ace1-119a7a953adc None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.357174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8c3d-36d5-46fa-ace1-119a7a953adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:05.357495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8c3d-36d5-46fa-ace1-119a7a953adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:05.358866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8c3d-36d5-46fa-ace1-119a7a953adc None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:05.370756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 282/567] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:05.371926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8bf02-f396-4fed-aa97-9948e6769f34 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.372207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8bf02-f396-4fed-aa97-9948e6769f34 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.372477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8bf02-f396-4fed-aa97-9948e6769f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.389715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-20c8bf02-f396-4fed-aa97-9948e6769f34 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:05.390887 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-20c8bf02-f396-4fed-aa97-9948e6769f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:05.407365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46661bff-2273-43b8-bbeb-8a579f0338da None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.409044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:05.442199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8ae73-ef10-4a3f-8aed-122d1156de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.443350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8ae73-ef10-4a3f-8aed-122d1156de3d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.443623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8ae73-ef10-4a3f-8aed-122d1156de3d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.443890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8ae73-ef10-4a3f-8aed-122d1156de3d None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.444577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8ae73-ef10-4a3f-8aed-122d1156de3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:05.444880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8ae73-ef10-4a3f-8aed-122d1156de3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:05.446285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8ae73-ef10-4a3f-8aed-122d1156de3d None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:05.457859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 283/568] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:05.458478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 286/569] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:05.458851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8bf02-f396-4fed-aa97-9948e6769f34 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.459709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f44c4-1cba-4264-9317-c815a807feef None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.460000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f44c4-1cba-4264-9317-c815a807feef None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.460271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f44c4-1cba-4264-9317-c815a807feef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.477871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-c07f44c4-1cba-4264-9317-c815a807feef None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:05.479074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c07f44c4-1cba-4264-9317-c815a807feef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:05.499693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:05.546802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 287/570] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:05.547572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f44c4-1cba-4264-9317-c815a807feef None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.567171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8bf02-f396-4fed-aa97-9948e6769f34 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.600343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.600977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.601251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.601549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.611320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:05.612746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:05.628005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:05.629003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:05.630557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:05.630557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:05.630557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:05.630557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:05.630557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 284/571] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:05.630557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.641384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:05.650588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f44c4-1cba-4264-9317-c815a807feef None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.684331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.684959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.685235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.685535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.691960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:05.694043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:05.705483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:05.706543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 288/572] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:05.707469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.725564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:05.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb437c60-4d63-4915-97e8-56f793ceaff6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.774460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.775099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.775385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.775663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.782305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:05.783753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:05.795483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:05.796527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:05.797490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 285/573] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:05.798021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.812045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50b7f4-01d8-46a4-9f39-9ca3ac160fb1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.846780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3a062-129e-4a74-a09b-b7e920dde10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.847420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a062-129e-4a74-a09b-b7e920dde10e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.847701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a062-129e-4a74-a09b-b7e920dde10e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.847982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a062-129e-4a74-a09b-b7e920dde10e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.857920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3a062-129e-4a74-a09b-b7e920dde10e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-96e3a062-129e-4a74-a09b-b7e920dde10e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:05.863238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:05.864910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 289/574] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:05.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3a062-129e-4a74-a09b-b7e920dde10e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.881274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:05.906862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a297-f803-4480-a886-6bfaedbc23a0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.939687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa8d39-1fd4-485e-9da7-11755f89e3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:05.940328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8d39-1fd4-485e-9da7-11755f89e3a0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:05.940611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8d39-1fd4-485e-9da7-11755f89e3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:05.940888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8d39-1fd4-485e-9da7-11755f89e3a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:05.951937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa8d39-1fd4-485e-9da7-11755f89e3a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-56fa8d39-1fd4-485e-9da7-11755f89e3a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:05.957217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:05.959018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:05.959881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 286/575] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:05.960565 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa8d39-1fd4-485e-9da7-11755f89e3a0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:05.970107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3a062-129e-4a74-a09b-b7e920dde10e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:05.970580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.004049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.005285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.005879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.006072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.006689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:06.006988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.008369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.020248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 290/576] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:06.021744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.059451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa8d39-1fd4-485e-9da7-11755f89e3a0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.090070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.092941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.093588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.093864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.094129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.094877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:06.095174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.096604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.108729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 287/577] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:06.109590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.127909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef829f99-41c2-41ee-a8da-412d0552e797 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.163109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.163733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.164014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.164293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.165078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:06.165431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.167014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.176216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.178966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 291/578] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:06.180149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.211906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d7f27-86f3-4560-911a-a376426803fe None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.246690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.247309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.247591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.247867 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.248571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:06.248875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.250283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.263345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 288/579] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:06.263345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.282134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f9c1-3d4f-4cc5-b93c-a4579622cadc None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.311002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.316050 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.316656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.316933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.317197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.318477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:06.318772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.320130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.327959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 292/580] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:06.328704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.366301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5027-3bee-413c-bd6b-7d745469d040 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.394422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.400321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.400955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.401231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.401530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.408788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.410307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:06.421508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:06.422563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 22:08:06.423576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:06.424346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 289/581] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:06.425281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.429223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c798-e17e-4b11-90aa-15651de91b12 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.462839 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5427c-4643-4468-8fb7-f90330003d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.463495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5427c-4643-4468-8fb7-f90330003d1d None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.463770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5427c-4643-4468-8fb7-f90330003d1d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.464044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5427c-4643-4468-8fb7-f90330003d1d None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/41d9317e24714de1a310198bb35eb00b/roles/7329827bdc90439cb95f7d079698b690` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.479155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5427c-4643-4468-8fb7-f90330003d1d None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=41d9317e24714de1a310198bb35eb00b, role_id=7329827bdc90439cb95f7d079698b690)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.480598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5427c-4643-4468-8fb7-f90330003d1d None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.494555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 293/582] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:06 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/41d9317e24714de1a310198bb35eb00b/roles/7329827bdc90439cb95f7d079698b690 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:06.495328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5427c-4643-4468-8fb7-f90330003d1d None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.535628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd425-c33b-42e1-97e4-a1d5e4248115 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.563585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.579536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.580486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.580998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.581440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.582432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=848151a99fb74f6b8f8210cfc49afeff) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:06.583044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.584822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.591193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 290/583] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/role_assignments?user.id=848151a99fb74f6b8f8210cfc49afeff&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:06.592269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.644004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5427c-4643-4468-8fb7-f90330003d1d None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.678907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.693335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.694606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.694888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.695331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.709346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.710893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.737907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0d25-f273-47b2-a8a3-f82fc7e44ec7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:06.738570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:06.740124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:06.741920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:06.741920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:06.741920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:06.741920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:06.741920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 294/584] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:06.741920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.778744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.779405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.779690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.779974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.792305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.794625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:06.812980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:06.814038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:06.814928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:06.814928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:06.814928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:06.814928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:06.814928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 291/585] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:06.815745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.883036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.887271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354f3ef-7dfb-421f-8f9d-79b308ddad6b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.926177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.927449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.927976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.928160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.928887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:06.929193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.930707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.930821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463c550-e64a-4892-b0c1-30a0cf73e526 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:06.938265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:06.953841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 295/586] 10.222.0.109 () {56 vars in 1142 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:06.955067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:06.968194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:06.968817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:06.969320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:06.969485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:06.970088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:06.970394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:06.971765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:06.985183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 292/587] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:06.986072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.072919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e76126-7e78-45a9-8d61-978eb05462b8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.117034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba4a54-ef52-4a9c-9411-ec7d73a26976 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.118291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.119447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.119907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.120384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.121258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:07.121771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.123337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.135775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 296/588] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:07.136893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.154351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:07.157220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.157909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.158217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.158716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.171574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.173205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:07.189817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:07.190791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:07.191675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:07.191675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:07.191675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:07.191675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:07.191675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 293/589] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:07.197312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.223625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:07.258353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223840a-7777-4286-a1ee-9b1b9bd7aaa9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.295857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.296913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.297402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.297965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] PATH_INFO: `/v3/users/glance` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.306321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.308039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.312669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca4c83-1f21-4f19-b1c3-cdf1bd023939 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:07.320956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:07.322061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 22:08:07.323066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:07.325666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 297/590] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:07.326676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.357177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.357853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.358137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.358427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.371900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.373605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:07.389353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:07.390331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 22:08:07.391263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:07.391805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 294/591] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:07.392624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.421071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:07.449472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e7fd1-e037-4ca9-ab4c-02b9f07d33ee None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.480998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:07.490345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.492086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.492579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.493096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.494118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:07.494629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.496255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.503587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0977f8-2b10-477a-ae52-f94a4ff97fa9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.510431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 298/592] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:07.511998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.541999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.542877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.543160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.543429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.544216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:07.544511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.546643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.569574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 295/593] 10.222.0.109 () {56 vars in 1142 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:07.570782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.627538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d484c42-6a3c-46aa-b7ac-4755cb329ebe None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.671248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.672303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.672589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.672915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.673707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:07.674388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.676207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.678670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:07.701583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 299/594] 10.222.0.109 () {56 vars in 1144 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:07.702500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.710333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551488a8-0d6e-4a62-b2f0-fd4a2218d3ee None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.756514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.757170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.757485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.758126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.761567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:07.768764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.770848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:07.788010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:07.788935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:07.789871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 296/595] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:07.790802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.831804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac15b-663d-4111-9c44-4d05f56cac23 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.889339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.890377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.891320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.891834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.908066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.910739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.918817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3e796-f720-4824-b5a1-937d3442e021 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:07.935461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:07.936366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:07.937250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:07.937250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:07.937250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:07.937250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:07.944570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 300/596] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:07.944570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:07.964907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:07.965597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:07.965879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:07.966150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:07.979888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:07.981603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:07.998691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:07.999566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:08.000519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:08.000519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:08.000519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:08.000519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:08.001472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:08.003987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 297/597] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:08.005923 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.062843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:08.068208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee38c-062f-455f-989c-4b6bdd4a5bb9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.104616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.105245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.105582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.105859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.106745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:08.107020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.108688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.122741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 301/598] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:08.123683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.127918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c961cfb-3f6b-4c22-923d-c42953581046 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.167415 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.168474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.168974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.169490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.170425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:08.170926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.172617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.193460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 298/599] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:08.194759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.231309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dab1e-f095-499b-ad20-a80e20ca364f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.233773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:08.269968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.271246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.271246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.271849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.272494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:08.272796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.274209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.291364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 302/600] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:08.292165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.313773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de6c3-4655-45cb-b047-f7300e114baf None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.326111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:08.355815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96fe10-6287-492d-a534-f2cd9c9236e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.356991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96fe10-6287-492d-a534-f2cd9c9236e4 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.357755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96fe10-6287-492d-a534-f2cd9c9236e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.358277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96fe10-6287-492d-a534-f2cd9c9236e4 None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/1d0422cd82114045a9fff7df29bc3906/roles/7329827bdc90439cb95f7d079698b690` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.376817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96fe10-6287-492d-a534-f2cd9c9236e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=1d0422cd82114045a9fff7df29bc3906, role_id=7329827bdc90439cb95f7d079698b690)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.378318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96fe10-6287-492d-a534-f2cd9c9236e4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.397947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 299/601] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:08 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/1d0422cd82114045a9fff7df29bc3906/roles/7329827bdc90439cb95f7d079698b690 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:08.399616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96fe10-6287-492d-a534-f2cd9c9236e4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.416043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694266f-5c95-4963-ae21-b9d88c93616f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.459866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.460536 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.460817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.461090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.469261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.471241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:08.484670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:08.485666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:08.487013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 303/602] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:08.493010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.523968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:08.553726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96fe10-6287-492d-a534-f2cd9c9236e4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.603617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.604909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.605395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.605908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.608978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c27068-a5d8-4e4f-82ef-f68f099861e1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.624019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.626011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:08.646421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:08.647374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:08.648256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 300/603] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:08.651461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed66d33-551f-4b2b-97f8-2eabf2cc792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.652098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed66d33-551f-4b2b-97f8-2eabf2cc792f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.652377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed66d33-551f-4b2b-97f8-2eabf2cc792f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.652648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed66d33-551f-4b2b-97f8-2eabf2cc792f None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.653503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed66d33-551f-4b2b-97f8-2eabf2cc792f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:08.653820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed66d33-551f-4b2b-97f8-2eabf2cc792f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.655273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed66d33-551f-4b2b-97f8-2eabf2cc792f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.658295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.674789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 304/604] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:08.676127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b6228-2d6d-4915-b93a-a1ec0e4b7505 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.676984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b6228-2d6d-4915-b93a-a1ec0e4b7505 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.677359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b6228-2d6d-4915-b93a-a1ec0e4b7505 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.684224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 305/605] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:08 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:08.688456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b6228-2d6d-4915-b93a-a1ec0e4b7505 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.695317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:08.792381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3076efd-8e05-46fb-b17b-de0def7165b1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.796354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:08.815440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b6228-2d6d-4915-b93a-a1ec0e4b7505 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:08.832435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8eccb-6603-424d-a954-31033df6f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.833888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8eccb-6603-424d-a954-31033df6f302 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.834377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8eccb-6603-424d-a954-31033df6f302 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.834659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8eccb-6603-424d-a954-31033df6f302 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.835576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8eccb-6603-424d-a954-31033df6f302 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:08.835886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8eccb-6603-424d-a954-31033df6f302 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.837505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8eccb-6603-424d-a954-31033df6f302 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.856456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 301/606] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:08.856456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58938a-2f47-40ae-91af-ed70cf9bfa87 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.856456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58938a-2f47-40ae-91af-ed70cf9bfa87 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.856456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58938a-2f47-40ae-91af-ed70cf9bfa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.859737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83a892-1593-4d25-8df0-a3da35967583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:08.860703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83a892-1593-4d25-8df0-a3da35967583 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.861247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83a892-1593-4d25-8df0-a3da35967583 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.861771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83a892-1593-4d25-8df0-a3da35967583 None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/33183e28de8f487f82f92f0151e7c73c/roles/7329827bdc90439cb95f7d079698b690` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.873804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ec58938a-2f47-40ae-91af-ed70cf9bfa87 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:08.875162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ec58938a-2f47-40ae-91af-ed70cf9bfa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:08.880280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83a892-1593-4d25-8df0-a3da35967583 None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=33183e28de8f487f82f92f0151e7c73c, role_id=7329827bdc90439cb95f7d079698b690)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:08.882135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83a892-1593-4d25-8df0-a3da35967583 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:08.902631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 306/607] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:08 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/33183e28de8f487f82f92f0151e7c73c/roles/7329827bdc90439cb95f7d079698b690 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:08.903952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83a892-1593-4d25-8df0-a3da35967583 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:08.966779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 302/608] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:08.972978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5b2f5-112b-47f5-a676-ea0e9677551e None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.972978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5b2f5-112b-47f5-a676-ea0e9677551e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.972978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5b2f5-112b-47f5-a676-ea0e9677551e None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.974577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 303/609] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:08 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:08.976108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81569a4-6330-41cc-bb6c-e21c84e464f5 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:08.976389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81569a4-6330-41cc-bb6c-e21c84e464f5 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:08.976659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81569a4-6330-41cc-bb6c-e21c84e464f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:08.996772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e81569a4-6330-41cc-bb6c-e21c84e464f5 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:08.998046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e81569a4-6330-41cc-bb6c-e21c84e464f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:09.000333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:09.070207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83a892-1593-4d25-8df0-a3da35967583 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:09.077131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 304/610] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:09.077870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81569a4-6330-41cc-bb6c-e21c84e464f5 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:09.117229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f2669-0324-4455-aec9-3a784838875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:09.118670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2669-0324-4455-aec9-3a784838875c None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:09.119109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2669-0324-4455-aec9-3a784838875c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:09.119663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f2669-0324-4455-aec9-3a784838875c None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/5455c117e349480ba8b8c020b767bec5/roles/7329827bdc90439cb95f7d079698b690` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:09.123182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:09.139360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f2669-0324-4455-aec9-3a784838875c None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=5455c117e349480ba8b8c020b767bec5, role_id=7329827bdc90439cb95f7d079698b690)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:09.140824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f2669-0324-4455-aec9-3a784838875c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:09.157338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 307/611] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:08 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/5455c117e349480ba8b8c020b767bec5/roles/7329827bdc90439cb95f7d079698b690 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:09.158322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f2669-0324-4455-aec9-3a784838875c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:09.211687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81569a4-6330-41cc-bb6c-e21c84e464f5 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:09.257335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c64296-6f11-4ea2-a694-1995781db8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:09.258449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c64296-6f11-4ea2-a694-1995781db8cf None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:09.259097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c64296-6f11-4ea2-a694-1995781db8cf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:09.259576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c64296-6f11-4ea2-a694-1995781db8cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:09.268487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:09.275386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c64296-6f11-4ea2-a694-1995781db8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:09.277760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f2669-0324-4455-aec9-3a784838875c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-57c64296-6f11-4ea2-a694-1995781db8cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:09.281457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:09.282427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:09.283283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:09.287023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 305/612] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:09.294606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c64296-6f11-4ea2-a694-1995781db8cf None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:09.315663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88656b-b7b7-4dae-b57f-3198a08ee494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:09.316311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88656b-b7b7-4dae-b57f-3198a08ee494 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:09.316586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88656b-b7b7-4dae-b57f-3198a08ee494 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:09.316855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88656b-b7b7-4dae-b57f-3198a08ee494 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:09.329301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88656b-b7b7-4dae-b57f-3198a08ee494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-4e88656b-b7b7-4dae-b57f-3198a08ee494 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:09.334329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:09.335312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:09.336906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 308/613] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:09.342058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88656b-b7b7-4dae-b57f-3198a08ee494 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:09.399119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:09.461993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88656b-b7b7-4dae-b57f-3198a08ee494 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:09.463690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c64296-6f11-4ea2-a694-1995781db8cf None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:09.500485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:09.501192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:09.501499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:09.501871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:09.502582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:09.502876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:09.504627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:09.507517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:09.508154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:09.508435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:09.508710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:09.509456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:09.509765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:09.511984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:09.513752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 309/614] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:09.524302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:09.526662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 306/615] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:09.532479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:09.602292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:09.650419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c07b6-a9cf-43d2-9dbc-ad3419838051 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:09.690180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:09.690845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:09.691133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:09.691413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:09.703432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:09.705108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:09.721626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:09.722541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:09.725121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:09.725121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:09.725121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:09.725121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:09.725121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 307/616] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:09.729850 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:09.799351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625335d-90a0-49c2-b9ae-ca6e9baa2517 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:09.814026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:09.844655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b4d72-ca42-4a0d-9002-94c75fccbbfe None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:09.869890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:09.889785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:09.890427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:09.891942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:09.892215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:09.892917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:09.893494 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:09.894874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:09.898433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:09.899338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:09.899832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:09.900319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:09.908395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 308/617] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:09.914832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:09.929802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:09.931797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:09.952576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:09.953540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:09.954428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:09.954428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:09.954428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:09.954428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:09.956704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 310/618] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:09.965065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:10.103861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f82ff-2f3e-4568-89d7-d46b1c923ef4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:10.159056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:10.159620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:10.159900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:10.160182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:10.171916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:10.173358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:10.188449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:10.189421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:10.191067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 309/619] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:10.195691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:10.198354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415952-7b38-4970-bc38-64c197920500 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:10.252496 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:10.273043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:10.273694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:10.273974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:10.274242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:10.274934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:10.275233 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:10.276612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:10.301927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 311/620] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:10.305141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:10.335568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:10.350466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36af65-1c0c-4f50-836c-dd4602057a1b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:10.389839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:10.390697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:10.391017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:10.391335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:10.392662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:10.392977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:10.397022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:10.416995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 310/621] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:08:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:10.426497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:10.469201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65485436-a64f-4e52-b52f-77d479a74959 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:10.532074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:10.532725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:10.533002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:10.533276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:10.546924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:10.548398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:10.561621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:10.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:10.564033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 312/622] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:10.569165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:10.591376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:10.620368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a46e1-40a0-43a0-bfae-fb88e486efad None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:10.667536 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:10.695761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbfc05-ae51-401e-8041-565dcd4600fd None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:10.712121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125759-08db-4b4e-926a-a2f7e919fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:10.713010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125759-08db-4b4e-926a-a2f7e919fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:10.713337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125759-08db-4b4e-926a-a2f7e919fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:10.713737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125759-08db-4b4e-926a-a2f7e919fcd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:10.734951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:10.735916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:10.736414 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:10.736884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:10.737923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:10.738442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:10.741460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:10.764719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 313/623] 10.222.0.109 () {56 vars in 1140 bytes} [Thu Aug 4 22:08:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:10.772764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:10.890957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a88ef-417c-472e-96eb-f3618ead1243 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:10.902753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01125759-08db-4b4e-926a-a2f7e919fcd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:10.904226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01125759-08db-4b4e-926a-a2f7e919fcd8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:10.935984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc91a5-02dc-407c-8fdb-c7febf305d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:10.936803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc91a5-02dc-407c-8fdb-c7febf305d00 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:10.937084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc91a5-02dc-407c-8fdb-c7febf305d00 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:10.937356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc91a5-02dc-407c-8fdb-c7febf305d00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:10.941058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:10.953694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc91a5-02dc-407c-8fdb-c7febf305d00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:10.955781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc91a5-02dc-407c-8fdb-c7febf305d00 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-ddfc91a5-02dc-407c-8fdb-c7febf305d00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:10.975829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:10.976937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:10.978052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:10.978052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:10.978052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:10.978052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:10.980585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 314/624] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:10.981946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601120a-a8a6-4f02-bcc5-99c56b5dda85 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:10.982386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601120a-a8a6-4f02-bcc5-99c56b5dda85 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:10.982794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601120a-a8a6-4f02-bcc5-99c56b5dda85 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:10.984807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 315/625] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:10.985811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601120a-a8a6-4f02-bcc5-99c56b5dda85 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.001851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:11.012900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 311/626] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:08:11.014660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25841-7358-4919-8a26-b3ee22c2e0d0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.014884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25841-7358-4919-8a26-b3ee22c2e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.015307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25841-7358-4919-8a26-b3ee22c2e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.034100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-83b25841-7358-4919-8a26-b3ee22c2e0d0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:11.035494 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-83b25841-7358-4919-8a26-b3ee22c2e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:11.116705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 312/627] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:11.116954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601120a-a8a6-4f02-bcc5-99c56b5dda85 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.117951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b25841-7358-4919-8a26-b3ee22c2e0d0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.155945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.156715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.157100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.157524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.158532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:11.158700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:11.160195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:11.173390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 316/628] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:11.174385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.231967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:11.247782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b25841-7358-4919-8a26-b3ee22c2e0d0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.278917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:11.296387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456672-a9d4-461a-92fe-4d38bb2bd0b2 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.297792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.298755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.299977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.300405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.313560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:11.315404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:11.334892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d061b-3e5c-4eb9-b221-ca4ef9a9a449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.335697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d061b-3e5c-4eb9-b221-ca4ef9a9a449 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:11.335854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:11.336928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:11.338000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:11.338000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:11.338000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:11.338000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:11.338241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d061b-3e5c-4eb9-b221-ca4ef9a9a449 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.338596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d061b-3e5c-4eb9-b221-ca4ef9a9a449 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.340007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 313/629] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:11.341187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.355770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d061b-3e5c-4eb9-b221-ca4ef9a9a449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-8b7d061b-3e5c-4eb9-b221-ca4ef9a9a449 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:11.361163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:11.362292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:11.363152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 317/630] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:11.364409 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d061b-3e5c-4eb9-b221-ca4ef9a9a449 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.469289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed340dde-c0d3-40e5-b1b9-9a4540fccfa6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.472560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d061b-3e5c-4eb9-b221-ca4ef9a9a449 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.484861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:11.508303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.508963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.509253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.509569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.512991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8c30a-921e-4f18-bfb2-975886a8e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.513822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c30a-921e-4f18-bfb2-975886a8e33a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.514265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c30a-921e-4f18-bfb2-975886a8e33a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.514675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c30a-921e-4f18-bfb2-975886a8e33a None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.515492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8c30a-921e-4f18-bfb2-975886a8e33a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:11.515933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8c30a-921e-4f18-bfb2-975886a8e33a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:11.517058 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:11.517647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8c30a-921e-4f18-bfb2-975886a8e33a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:11.518796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:11.530486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 318/631] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:11.530652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:11.531493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:11.532359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 314/632] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:11.532949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.534258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05824c8-f2aa-4063-bd23-6b6f119d1671 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.534683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05824c8-f2aa-4063-bd23-6b6f119d1671 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.535132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05824c8-f2aa-4063-bd23-6b6f119d1671 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.536630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 319/633] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:11.538447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05824c8-f2aa-4063-bd23-6b6f119d1671 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.572566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:11.648188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb2a14-5be5-4687-b349-5b9566e497b7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.657968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05824c8-f2aa-4063-bd23-6b6f119d1671 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.684621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd922249-1b86-4710-9e16-3ed88ddc17cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.685254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd922249-1b86-4710-9e16-3ed88ddc17cd None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.685567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd922249-1b86-4710-9e16-3ed88ddc17cd None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.685843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd922249-1b86-4710-9e16-3ed88ddc17cd None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.686553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd922249-1b86-4710-9e16-3ed88ddc17cd None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:11.686857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd922249-1b86-4710-9e16-3ed88ddc17cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:11.688220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd922249-1b86-4710-9e16-3ed88ddc17cd None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:11.695087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.695787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.696076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.696351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.697102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:11.697368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 315/634] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:11.698584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb007-dcee-45e4-93dd-8dc54a0824b8 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.698865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb007-dcee-45e4-93dd-8dc54a0824b8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.699144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb007-dcee-45e4-93dd-8dc54a0824b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.702119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:11.703608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:11.717522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e8fcb007-dcee-45e4-93dd-8dc54a0824b8 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:11.717866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 320/635] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:11.722698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e8fcb007-dcee-45e4-93dd-8dc54a0824b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:11.722794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.750676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:11.788428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 316/636] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:11.789271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb007-dcee-45e4-93dd-8dc54a0824b8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.815692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:11.828554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd8d7b-132b-4c53-9ae2-0cf1fd5afb99 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.861622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16ea1f-7741-4aef-b625-dc6e93ec709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.862353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ea1f-7741-4aef-b625-dc6e93ec709a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.862635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ea1f-7741-4aef-b625-dc6e93ec709a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.862903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ea1f-7741-4aef-b625-dc6e93ec709a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.870041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16ea1f-7741-4aef-b625-dc6e93ec709a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:11.871469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16ea1f-7741-4aef-b625-dc6e93ec709a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-1f16ea1f-7741-4aef-b625-dc6e93ec709a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:11.882719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:11.883784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:11.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 321/637] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:11.885250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42af8-1f60-47b4-8f39-749da3cad3e2 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.885250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42af8-1f60-47b4-8f39-749da3cad3e2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.885609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a42af8-1f60-47b4-8f39-749da3cad3e2 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.886722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 322/638] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:11 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:11.887394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a42af8-1f60-47b4-8f39-749da3cad3e2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.897435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb007-dcee-45e4-93dd-8dc54a0824b8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:11.932824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:11.933512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:11.933836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:11.934170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:11.944323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:11.945806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:11.960328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:11.961382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:11.962336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:11.962336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:11.962336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:11.962336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:11.962336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 317/639] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:11.962336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:11.975217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:11.990036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a42af8-1f60-47b4-8f39-749da3cad3e2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.023905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcd462-4a7b-4c01-8ab8-5bc95e43af8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.024078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcd462-4a7b-4c01-8ab8-5bc95e43af8a None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.024353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcd462-4a7b-4c01-8ab8-5bc95e43af8a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.024631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcd462-4a7b-4c01-8ab8-5bc95e43af8a None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/848151a99fb74f6b8f8210cfc49afeff/roles/7329827bdc90439cb95f7d079698b690` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.039368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcd462-4a7b-4c01-8ab8-5bc95e43af8a None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=848151a99fb74f6b8f8210cfc49afeff, role_id=7329827bdc90439cb95f7d079698b690)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:12.040821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcd462-4a7b-4c01-8ab8-5bc95e43af8a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:12.045389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:12.055544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 323/640] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:11 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/848151a99fb74f6b8f8210cfc49afeff/roles/7329827bdc90439cb95f7d079698b690 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:12.056341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcd462-4a7b-4c01-8ab8-5bc95e43af8a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.072851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5f724-8924-41a0-ac11-e553f183a2d2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.115809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c43ce-6bda-4e1f-9170-970665450a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.116491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c43ce-6bda-4e1f-9170-970665450a43 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.116767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c43ce-6bda-4e1f-9170-970665450a43 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.117032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c43ce-6bda-4e1f-9170-970665450a43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.129521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c43ce-6bda-4e1f-9170-970665450a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-a46c43ce-6bda-4e1f-9170-970665450a43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:12.135281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:12.136327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:12.137181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 318/641] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:12.137910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf6a61-2e80-4ad4-9dbd-bcfeb8b1cca8 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.138539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf6a61-2e80-4ad4-9dbd-bcfeb8b1cca8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.138539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf6a61-2e80-4ad4-9dbd-bcfeb8b1cca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.158773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9dcf6a61-2e80-4ad4-9dbd-bcfeb8b1cca8 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:12.160000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9dcf6a61-2e80-4ad4-9dbd-bcfeb8b1cca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:12.183061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcd462-4a7b-4c01-8ab8-5bc95e43af8a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.225030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.225936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.226061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.226222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.226983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:12.227289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:12.228740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:12.237668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 319/642] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:12.238219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:12.238677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf6a61-2e80-4ad4-9dbd-bcfeb8b1cca8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.255278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 324/643] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:12.256213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.318879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:12.358234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf6a61-2e80-4ad4-9dbd-bcfeb8b1cca8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.365682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ad248-2fe2-45e6-a743-dd07caedb6c7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.402284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.402912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.403187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.403457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.404162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:12.404349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.404461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:12.404978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.405260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.405568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.405861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:12.406263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:12.406571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:12.408192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:12.417715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 325/644] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:12.418436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.426181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 320/645] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:12.426181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.494537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:12.540105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a0964-2c88-4832-ac26-eb4016f56d07 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.541847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf2906-fd28-44f0-858d-00c265755a28 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.567876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:12.577297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1d939-a957-48e2-b7d8-86843b40dc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.577903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1d939-a957-48e2-b7d8-86843b40dc32 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.578178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1d939-a957-48e2-b7d8-86843b40dc32 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.578446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1d939-a957-48e2-b7d8-86843b40dc32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.583271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.583980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.584260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.584462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.585244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=41d9317e24714de1a310198bb35eb00b) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:12.585584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:12.587078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:12.589969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1d939-a957-48e2-b7d8-86843b40dc32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:12.593061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 326/646] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/role_assignments?user.id=41d9317e24714de1a310198bb35eb00b&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 610 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:12.594133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-9ff1d939-a957-48e2-b7d8-86843b40dc32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:12.595528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:12.596460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:12.597869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 321/647] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:12.598769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1d939-a957-48e2-b7d8-86843b40dc32 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.734337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1d939-a957-48e2-b7d8-86843b40dc32 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.740108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406928-0f8f-427f-b165-8995d2c460ca None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.772370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:12.779282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.779282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.779567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.779883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.788946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:12.789119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.789669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.790113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.790436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.790538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:12.803194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:12.804236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 322/648] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:12.805898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.811174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:12.812676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:12.830694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:12.831732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:12.833481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:12.833481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:12.833481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:12.833481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:12.833481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 327/649] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:12.833481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:12.875994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:12.938669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b1af-ded0-4893-97c8-616021a5ccea None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.975801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07fac-e026-41cc-90f1-bb57a3dcd837 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:12.976445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:12.977302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:12.977886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:12.978354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:12.979306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:12.979852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:12.981432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:12.991472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 323/650] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:12.992702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.015722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.016339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.016615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.016885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.017708 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:13.018013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.020682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.041687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 328/651] 10.222.0.109 () {56 vars in 1142 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:13.043186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.098004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:13.123229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cb84-d2cd-4804-be47-895eb70598e5 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:13.135366 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:13.171461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.172101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.172386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.172806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.173544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:13.174127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.175528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.194280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b2366-aa69-4f8f-bb30-ce2e45f582ce None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:13.197680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 324/652] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:13.200104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.235062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.236110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.236606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.237081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.249282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.251592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:13.268768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:13.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:13.271015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:13.271015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:13.271015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:13.271015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:13.272828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 329/653] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:13.273990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.322653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a12889-04f1-4eb9-8d44-627de1c380a1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:13.360609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227fe4f-0bb7-4596-8559-9b5d238323a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.361012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227fe4f-0bb7-4596-8559-9b5d238323a7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.361293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227fe4f-0bb7-4596-8559-9b5d238323a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.361611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227fe4f-0bb7-4596-8559-9b5d238323a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.378179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:13.396398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504669f-4808-41c1-8fff-bfc0d30bd26c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:13.401664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:13.435410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.436432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.436943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.437469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] PATH_INFO: `/v3/users/glance` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.439780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227fe4f-0bb7-4596-8559-9b5d238323a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.441362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227fe4f-0bb7-4596-8559-9b5d238323a7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.447059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.448880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:13.462366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:13.463352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 22:08:13.464313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:13.467204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 330/654] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:13.468379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.558644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 325/655] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:08:13.559648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227fe4f-0bb7-4596-8559-9b5d238323a7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.579646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da9d1-6293-4be2-b9e0-5d06b90682ba None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:13.617683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.618646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.619129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.619635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.620558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:13.621059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.622743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.633870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:13.636283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 331/656] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:13.637702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.649469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:13.674733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227fe4f-0bb7-4596-8559-9b5d238323a7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:13.712519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.713398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.713883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.714313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.715374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:13.715825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.717565 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.744673 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 326/657] 10.222.0.109 () {56 vars in 1142 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:13.745681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.759638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fd914-54b1-4984-bf8d-eed6f4b42396 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:13.806817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.807618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.808119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.808458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.820775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.822483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:13.839008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:13.840170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:13.841267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:13.841267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:13.841267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:13.841267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:13.842874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 332/658] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:13.843855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.872949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dcd86-ac55-4866-a056-a70662bdd00b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:13.911237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:13.911870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:13.913358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:13.913821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:13.914248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:13.917784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:13.924218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:13.926594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:13.940296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:13.941507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 22:08:13.943339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:13.949895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 327/659] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:13.951032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:13.963899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc5710-72fb-4168-9abc-03596113bbdb None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.000874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a1e5b-bf09-40fc-8327-8f169aab975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.001672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a1e5b-bf09-40fc-8327-8f169aab975b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.002241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a1e5b-bf09-40fc-8327-8f169aab975b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.002387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a1e5b-bf09-40fc-8327-8f169aab975b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.059589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071f831-349c-4ddb-b7b5-e8d623f26187 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.076366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a1e5b-bf09-40fc-8327-8f169aab975b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:14.078008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a1e5b-bf09-40fc-8327-8f169aab975b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:14.097386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.098389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.098845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.099267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.100102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:14.100555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:14.102212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:14.115144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 328/660] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:14.115823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.158045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:14.165460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:14.219516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 333/661] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:13 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:08:14.219516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900413-188a-4816-b524-c5afcd696af5 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.219516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900413-188a-4816-b524-c5afcd696af5 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.219516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900413-188a-4816-b524-c5afcd696af5 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.219516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 334/662] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:14.219516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67900413-188a-4816-b524-c5afcd696af5 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.233874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb5952-c2fc-4edc-ac4f-71ca5f2bc3c9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.269951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.271121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.271455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.271787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.272742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:14.273052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:14.274511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:14.293006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 329/663] 10.222.0.109 () {56 vars in 1144 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:14.293865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.342052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67900413-188a-4816-b524-c5afcd696af5 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.377717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdab54-7f89-40c8-adbe-942c2aca18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.378605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdab54-7f89-40c8-adbe-942c2aca18b5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.378880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdab54-7f89-40c8-adbe-942c2aca18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.379143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdab54-7f89-40c8-adbe-942c2aca18b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.386560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdab54-7f89-40c8-adbe-942c2aca18b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:14.387992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdab54-7f89-40c8-adbe-942c2aca18b5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-5efdab54-7f89-40c8-adbe-942c2aca18b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:14.399736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:14.400821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:14.402061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:14.402061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:14.402061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:14.402061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:14.402061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:14.402061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:14.402061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:14.402061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:14.405528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 335/664] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:14.406751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3a6ba-5c6a-4d63-860d-f168620645fa None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.407029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3a6ba-5c6a-4d63-860d-f168620645fa None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.407253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf94c4a-15e9-4d93-be87-f02ab3068372 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.407388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3a6ba-5c6a-4d63-860d-f168620645fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.414163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:14.425485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-34e3a6ba-5c6a-4d63-860d-f168620645fa None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:14.427433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-34e3a6ba-5c6a-4d63-860d-f168620645fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:14.442448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:14.451386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.451936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.452224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.452493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.463342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:14.464794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:14.481067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:14.482029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:14.482911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:14.482911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:14.482911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:14.482911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:14.482911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 330/665] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:14.483179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.507702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 336/666] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:14.514559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3a6ba-5c6a-4d63-860d-f168620645fa None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.595988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239676-c93c-4a25-ab0d-7724095c1941 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.630668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3a6ba-5c6a-4d63-860d-f168620645fa None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.632135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f541c-ba14-4f3f-bba5-7eb20e9596b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.632830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f541c-ba14-4f3f-bba5-7eb20e9596b5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.633169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f541c-ba14-4f3f-bba5-7eb20e9596b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.633542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f541c-ba14-4f3f-bba5-7eb20e9596b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.659291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:14.668223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.668786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.669063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.669437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.670115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:14.671086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:14.671896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:14.688616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 337/667] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:14.688616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.701279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:14.715089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f541c-ba14-4f3f-bba5-7eb20e9596b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:14.716542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f541c-ba14-4f3f-bba5-7eb20e9596b5 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:14.799538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f1b49-8345-4974-9220-50761ef78ca5 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.822657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 331/668] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:08:14.823523 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f541c-ba14-4f3f-bba5-7eb20e9596b5 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.840653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.841274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.841586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.841858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.842548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:14.842847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:14.844215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:14.857849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 338/669] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:14.858738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.882638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:14.924515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:14.927561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f541c-ba14-4f3f-bba5-7eb20e9596b5 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.957885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0538dd-d28f-4617-b811-0f1d94cba946 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:14.961171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78f2d-1c88-4690-8b7b-f36470bfbf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:14.961842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78f2d-1c88-4690-8b7b-f36470bfbf6e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.962119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78f2d-1c88-4690-8b7b-f36470bfbf6e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.962388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78f2d-1c88-4690-8b7b-f36470bfbf6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.972209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c78f2d-1c88-4690-8b7b-f36470bfbf6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-38c78f2d-1c88-4690-8b7b-f36470bfbf6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:14.978106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:14.979104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:14.979988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 332/670] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:14.980718 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee111e5-e2b6-4770-b19d-f2c4ad05295a None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:14.981041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee111e5-e2b6-4770-b19d-f2c4ad05295a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:14.981239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee111e5-e2b6-4770-b19d-f2c4ad05295a None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:14.982522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 333/671] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:14.983213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee111e5-e2b6-4770-b19d-f2c4ad05295a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:14.999800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.000412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.000689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.000957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.001769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=1d0422cd82114045a9fff7df29bc3906) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:15.002074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.003516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.009351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 339/672] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/role_assignments?user.id=1d0422cd82114045a9fff7df29bc3906&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 610 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:15.010211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.100575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee111e5-e2b6-4770-b19d-f2c4ad05295a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.122403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:15.131967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab026b5f-f6f9-4157-94d6-d5ed9a61d48c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.145621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.146286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.146558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.146820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.159051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.161504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.167346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:15.174608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9f837-63f9-46e7-8efa-a8844c6cd65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.175454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f837-63f9-46e7-8efa-a8844c6cd65b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.175729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f837-63f9-46e7-8efa-a8844c6cd65b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.176060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f837-63f9-46e7-8efa-a8844c6cd65b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:15.178475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:15.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:15.181765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:15.181765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:15.181765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:15.181765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:15.181765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 334/673] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:15.181765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.183622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9f837-63f9-46e7-8efa-a8844c6cd65b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.185696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9f837-63f9-46e7-8efa-a8844c6cd65b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-5af9f837-63f9-46e7-8efa-a8844c6cd65b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:15.197512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:15.198751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:15.200220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 340/674] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:15.204452 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1f190-26c1-4938-923e-bf645d868eaf None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.204769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1f190-26c1-4938-923e-bf645d868eaf None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.205049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1f190-26c1-4938-923e-bf645d868eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.224153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-35a1f190-26c1-4938-923e-bf645d868eaf None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:15.225338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-35a1f190-26c1-4938-923e-bf645d868eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:15.304305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 341/675] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:15.305159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1f190-26c1-4938-923e-bf645d868eaf None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.305632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cc85d-2f89-4c88-b251-aaab6febf8d0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.351236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.351864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.352145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.352416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.353129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:15.353450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.354824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.367881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 335/676] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:15.367881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.400721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:15.429337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1f190-26c1-4938-923e-bf645d868eaf None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.452756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:15.465934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.466559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.466835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.467097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.467869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:15.468173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.469563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.476406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e2233-a09a-4378-a023-8d183bb12258 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.482312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 342/677] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:15.483117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.514203 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.515071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.515359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.515736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.516388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:15.516850 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.518440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.532077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 336/678] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:15.532926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.603755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ac008-8d89-4b25-9f4c-97ee2ab9b742 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.636456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:15.639916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cb6a6-87c3-4e3a-a904-22ee6b25983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.640594 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cb6a6-87c3-4e3a-a904-22ee6b25983c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.640851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cb6a6-87c3-4e3a-a904-22ee6b25983c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.641118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cb6a6-87c3-4e3a-a904-22ee6b25983c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.650011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d7ef5-fa83-4c1f-a733-3a5acd0556e1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.653023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cb6a6-87c3-4e3a-a904-22ee6b25983c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-529cb6a6-87c3-4e3a-a904-22ee6b25983c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:15.659882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:15.660842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:15.661748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:15.662393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 343/679] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:15.662943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cb6a6-87c3-4e3a-a904-22ee6b25983c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.690119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.691002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.691503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.692209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.702598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.704038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.710094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:15.721901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:15.723489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:15.724623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:15.724623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:15.724623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:15.724623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:15.724623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 337/680] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:15.724909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.780591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cb6a6-87c3-4e3a-a904-22ee6b25983c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.815959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.817388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.817852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.818280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.826253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.827853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.830924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88585e5-fb19-4be3-bcb6-80eef9128145 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:15.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:15.840648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:15.841731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:15.841731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:15.841731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:15.841731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:15.841731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:15.841731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:15.841731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:15.841731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:15.843865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 344/681] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:15.844781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.871548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:15.872168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:15.872443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:15.872706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:15.873515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=33183e28de8f487f82f92f0151e7c73c) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:15.873847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:15.875277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:15.881835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 338/682] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/role_assignments?user.id=33183e28de8f487f82f92f0151e7c73c&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 610 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:15.886452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:15.900914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:15.970472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca921cd-125f-4402-b305-4d0d84d818d6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:15.972617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:16.012243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.013231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.013735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.014250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.015343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:16.015886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.017543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.032497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed0ad4-a8e2-4e53-88cd-90835e7bf8c9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.033161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 345/683] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:16.034363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.071573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.072193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.072471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.072741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.074543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:16.074847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.076243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.090905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 339/684] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:16.091757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.170044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad86a59-db4b-49f7-af87-224750800e2e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.190214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:16.217674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.218471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.218847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.219213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.220632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:16.220988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.223752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.233107 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ea6c4-67e3-4347-b9c6-85dc8024f81a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.240460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 346/685] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:16.246885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.274252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.275049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.275325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.275593 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.278998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:16.288682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.290269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:16.305998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:16.306939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:16.307814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:16.307814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:16.307814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:16.307814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:16.307814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 340/686] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:16.308532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.371701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb8c29-a2d5-42fa-8322-c0d43433827a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.408607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.409638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.410214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.410839 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.414109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6e27d-1054-4b15-b4d6-4a5782dbbc0e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.418753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.420430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.432613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:16.435909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:16.437203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:16.438155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:16.442419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 347/687] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:16.447655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.451552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.452840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.452840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.452840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.453517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=5455c117e349480ba8b8c020b767bec5) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:16.453871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.455314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.465536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 341/688] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/role_assignments?user.id=5455c117e349480ba8b8c020b767bec5&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 610 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:16.466404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.549942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:16.564265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44fd23-400a-41bf-8a15-6e7624fbfee2 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.605256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.607741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.608301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.608791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.609976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:16.611302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.616786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.636943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 348/689] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:16.642923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.703603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b2c8-8f36-4150-b4c6-216f8b18e43b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.748336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.748960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.749238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.749535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.750387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:16.750695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.752168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.771949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 342/690] 10.222.0.109 () {56 vars in 1140 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:16.777722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.795403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eab8a-3d81-4e23-8cf0-63d9b33ab450 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.834598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.835247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.835520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.835787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:16.836117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:16.846043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:16.847548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:16.860592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:16.861619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:16.862652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:16.863194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 349/691] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:16.870544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:16.878505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:16.910493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87487321-d3fd-40e5-93e7-a747fd9d978a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:16.977436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a603-edb4-4349-9e5b-b2c63556a399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:16.978064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a603-edb4-4349-9e5b-b2c63556a399 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:16.978344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a603-edb4-4349-9e5b-b2c63556a399 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:16.978617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a603-edb4-4349-9e5b-b2c63556a399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.058237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dd6a6-3cb0-4794-81a6-b204c93e57e0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:17.058819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a603-edb4-4349-9e5b-b2c63556a399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.060295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a603-edb4-4349-9e5b-b2c63556a399 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:17.112900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:17.116894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:17.117728 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.118112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.118475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.119451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:17.119842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.121388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:17.145112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 350/692] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:17.147144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:17.178516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 343/693] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:16 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 22:08:17.184174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a603-edb4-4349-9e5b-b2c63556a399 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:17.216408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:17.259381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc8b36-66d5-42c1-aa3c-b72131b948eb None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:17.290234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a603-edb4-4349-9e5b-b2c63556a399 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:17.302932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520439ae-db72-4468-9216-4bc460fac93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:17.303685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520439ae-db72-4468-9216-4bc460fac93a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.304042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520439ae-db72-4468-9216-4bc460fac93a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.304391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520439ae-db72-4468-9216-4bc460fac93a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.330925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754d8b4-f719-495c-a5a6-69a6da0beda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:17.331573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754d8b4-f719-495c-a5a6-69a6da0beda0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.331850 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754d8b4-f719-495c-a5a6-69a6da0beda0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.332300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754d8b4-f719-495c-a5a6-69a6da0beda0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.341997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:17.344106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754d8b4-f719-495c-a5a6-69a6da0beda0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.345743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754d8b4-f719-495c-a5a6-69a6da0beda0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-3754d8b4-f719-495c-a5a6-69a6da0beda0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:17.361518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:17.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:17.364376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:17.364376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:17.364376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:17.364376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:17.369446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 344/694] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:17.370873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378391-1575-43d3-99c1-02a180ec0ab6 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.371156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378391-1575-43d3-99c1-02a180ec0ab6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.371429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378391-1575-43d3-99c1-02a180ec0ab6 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.372723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 345/695] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:17 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:17.373435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb378391-1575-43d3-99c1-02a180ec0ab6 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:17.383136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520439ae-db72-4468-9216-4bc460fac93a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.384659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520439ae-db72-4468-9216-4bc460fac93a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:17.449274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:17.485303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb378391-1575-43d3-99c1-02a180ec0ab6 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:17.498495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 351/696] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:17 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 22:08:17.500212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7656a-7b0b-470c-9211-fc1ece5f20a2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.500646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7656a-7b0b-470c-9211-fc1ece5f20a2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.501113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7656a-7b0b-470c-9211-fc1ece5f20a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.521001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2ed7656a-7b0b-470c-9211-fc1ece5f20a2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:17.522519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:17.522745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2ed7656a-7b0b-470c-9211-fc1ece5f20a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:17.523149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.523430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.523704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.524397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:17.524702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.526556 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:17.539581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 346/697] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:17.540288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:17.601422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 352/698] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:17.602606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7656a-7b0b-470c-9211-fc1ece5f20a2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:17.629542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:17.653798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2590495-c478-44f4-befd-e9adf219142c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:17.690868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:17.691603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.692381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.692696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.702962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.704409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:17.717089 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:17.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:17.720692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:17.721961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:17.721961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:17.721961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:17.721961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:17.721961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 347/699] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:17.730355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:17.731746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7656a-7b0b-470c-9211-fc1ece5f20a2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:17.777230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:17.777914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.778194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.778467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.789546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.791302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:17.807804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:17.808905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:17.809852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:17.809852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:17.809852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:17.809852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:17.809852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:17.809852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:17.809852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:17.809852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:17.813842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 353/700] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:17.814688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:17.854245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b26bf-898f-40d6-8a4e-eb1dc1812970 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:17.858101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:17.901038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:17.901649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.901932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.902314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.902950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.909995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:17.917282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:17.918373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:17.919594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 348/701] 10.222.0.109 () {56 vars in 1134 bytes} [Thu Aug 4 22:08:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:17.920422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:17.955008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c4f3-9924-4ba1-a894-d312005d8fb4 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:17.995528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:17.996163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:17.996449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:17.996726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:17.997467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:17.997848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:17.999368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.017725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 354/702] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:18.018989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:18.031543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caa530-8f32-4cdc-a311-8d758f6a0545 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:18.035337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:18.071756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:18.072633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.073113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.073646 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.074672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:18.075128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:18.076708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.090583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 349/703] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:18.091833 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:18.138028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:18.145511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893d6dc-a8be-4a95-ac37-438a451ad6b0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:18.181531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:18.182505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.182993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.183489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.184365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:18.184903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:18.186545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.191448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 355/704] 10.222.0.109 () {56 vars in 1129 bytes} [Thu Aug 4 22:08:18 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:18.192812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:18.226475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22526eed-2473-44fd-8de0-3985290357cc None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:18.263965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:18.264612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.264889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.265157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.272509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:18.273980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.276908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:18.286489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:18.287431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 350/705] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:18.293644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:18.309465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def48d0e-d997-463c-8ec3-e507283b23a2 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:18.347206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83126d-8694-4362-801a-b1b9c0180d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:18.348509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83126d-8694-4362-801a-b1b9c0180d65 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.349045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83126d-8694-4362-801a-b1b9c0180d65 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.349163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83126d-8694-4362-801a-b1b9c0180d65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.349886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83126d-8694-4362-801a-b1b9c0180d65 None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=848151a99fb74f6b8f8210cfc49afeff) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:18.350197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83126d-8694-4362-801a-b1b9c0180d65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:18.351933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83126d-8694-4362-801a-b1b9c0180d65 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.358219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 356/706] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:18 2022] GET /identity/v3/role_assignments?user.id=848151a99fb74f6b8f8210cfc49afeff&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 610 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:18.365034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7a96-9534-41c3-91f6-d16662a2b563 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.365034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7a96-9534-41c3-91f6-d16662a2b563 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.365034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7a96-9534-41c3-91f6-d16662a2b563 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.365034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 357/707] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:18.365034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc7a96-9534-41c3-91f6-d16662a2b563 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:18.398413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:18.414571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bff04-a6b4-4e1f-a39a-b72573948711 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:18.464907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cbe1c-dd9b-4a26-af07-7409a70f41d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:18.465756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbe1c-dd9b-4a26-af07-7409a70f41d3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.466035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbe1c-dd9b-4a26-af07-7409a70f41d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.466305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbe1c-dd9b-4a26-af07-7409a70f41d3 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.467015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cbe1c-dd9b-4a26-af07-7409a70f41d3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:18.467469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cbe1c-dd9b-4a26-af07-7409a70f41d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:18.469256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cbe1c-dd9b-4a26-af07-7409a70f41d3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.478552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 351/708] 10.222.0.109 () {56 vars in 1129 bytes} [Thu Aug 4 22:08:18 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:18.480510 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16de162-a42e-4a10-990b-176ec5fdd962 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.480797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16de162-a42e-4a10-990b-176ec5fdd962 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.481074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16de162-a42e-4a10-990b-176ec5fdd962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.486921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc7a96-9534-41c3-91f6-d16662a2b563 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:18.505797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-d16de162-a42e-4a10-990b-176ec5fdd962 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:18.507213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d16de162-a42e-4a10-990b-176ec5fdd962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:18.542060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:18.544023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.544299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.544569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.545350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:18.545695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:18.551195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.568653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:18.580371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 358/709] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:18.582740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:18.633178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 352/710] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:18.660880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16de162-a42e-4a10-990b-176ec5fdd962 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:18.733953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac95eb8-1e49-4354-80c0-8c98679e5445 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:18.771241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:18.782798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:18.783938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.784368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.785004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.787351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=41d9317e24714de1a310198bb35eb00b) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:18.787822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:18.790157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.796614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16de162-a42e-4a10-990b-176ec5fdd962 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:18.801859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 359/711] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:18 2022] GET /identity/v3/role_assignments?user.id=41d9317e24714de1a310198bb35eb00b&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:18.849854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e90e92-8972-4392-ae59-b200bd46f8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:18.850903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90e92-8972-4392-ae59-b200bd46f8da None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:18.851411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90e92-8972-4392-ae59-b200bd46f8da None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:18.851882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90e92-8972-4392-ae59-b200bd46f8da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:18.870395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e90e92-8972-4392-ae59-b200bd46f8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:18.872664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e90e92-8972-4392-ae59-b200bd46f8da None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-27e90e92-8972-4392-ae59-b200bd46f8da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:18.894443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:18.896692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:18.897650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:18.897650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:18.897650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:18.897650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:18.900028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 353/712] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:18.906935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:18.981479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:19.025398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d709ca9-0d4a-4aa2-a8e9-e08bfdfc94fd None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:19.042997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee68ce-e75f-4fd4-9088-61486201942a None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:19.043313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee68ce-e75f-4fd4-9088-61486201942a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:19.043546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee68ce-e75f-4fd4-9088-61486201942a None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:19.045020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 354/713] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:19.050331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25468bf0-d7b3-4383-bc26-c195a867ef28 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:19.051410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25468bf0-d7b3-4383-bc26-c195a867ef28 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:19.051894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25468bf0-d7b3-4383-bc26-c195a867ef28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:19.067125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:19.067754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:19.068032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:19.068304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:19.068999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:19.069299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:19.070718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:19.078369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-25468bf0-d7b3-4383-bc26-c195a867ef28 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:19.079859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-25468bf0-d7b3-4383-bc26-c195a867ef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:19.086433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 360/714] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:19.098518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:19.186023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 355/715] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:08:19.210796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25468bf0-d7b3-4383-bc26-c195a867ef28 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:19.292259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bdd31-1c3e-4188-af86-4586e1d04d26 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:19.326067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:19.339119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce577497-d55b-4af6-924b-66335ab7fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:19.339742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce577497-d55b-4af6-924b-66335ab7fcbc None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:19.340020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce577497-d55b-4af6-924b-66335ab7fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:19.340293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce577497-d55b-4af6-924b-66335ab7fcbc None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:19.340985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce577497-d55b-4af6-924b-66335ab7fcbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:19.342457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce577497-d55b-4af6-924b-66335ab7fcbc None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:19.349767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:19.374056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 361/716] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:08:19 2022] POST /identity/v3/users => generated 255 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:19.420101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25468bf0-d7b3-4383-bc26-c195a867ef28 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:19.467315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:19.468122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:19.468414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:19.468686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] PATH_INFO: `/v3/services/network` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:19.469378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:19.470926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:19.482318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:19.483344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:19.484304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 356/717] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:19 2022] GET /identity/v3/services/network => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:19.491038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce577497-d55b-4af6-924b-66335ab7fcbc None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:19.727256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce577497-d55b-4af6-924b-66335ab7fcbc None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:19.762977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453a801-996b-4b6d-84b4-ac062c63b31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:19.763598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453a801-996b-4b6d-84b4-ac062c63b31d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:19.763871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453a801-996b-4b6d-84b4-ac062c63b31d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:19.764136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453a801-996b-4b6d-84b4-ac062c63b31d None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:19.764806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453a801-996b-4b6d-84b4-ac062c63b31d None admin] RBAC: Adding query filter params (name=network) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:19.765099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453a801-996b-4b6d-84b4-ac062c63b31d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:19.766496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453a801-996b-4b6d-84b4-ac062c63b31d None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:19.772823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 362/718] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:19 2022] GET /identity/v3/services?name=network => generated 125 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:19.778317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:19.885373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd8841-bfc5-4e3f-b5b3-01c29b9e8210 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:19.916412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:19.924656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6b6d3-4cf6-4d77-bf45-dfc0fbf99b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:19.925283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6b6d3-4cf6-4d77-bf45-dfc0fbf99b26 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:19.925589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6b6d3-4cf6-4d77-bf45-dfc0fbf99b26 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:19.926562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6b6d3-4cf6-4d77-bf45-dfc0fbf99b26 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:19.927193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6b6d3-4cf6-4d77-bf45-dfc0fbf99b26 None admin] RBAC: Adding query filter params (type=network) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:19.927891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6b6d3-4cf6-4d77-bf45-dfc0fbf99b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:19.929263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6b6d3-4cf6-4d77-bf45-dfc0fbf99b26 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:19.934583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 357/719] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:19 2022] GET /identity/v3/services?type=network => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:20.875960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d46e5d-c6c0-4b0f-9f76-0573e1bfe29f None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:20.876548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d46e5d-c6c0-4b0f-9f76-0573e1bfe29f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:20.876548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d46e5d-c6c0-4b0f-9f76-0573e1bfe29f None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:20.878124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 363/720] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:20.895073 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cd433-90d5-4c0a-b81e-dad939998ed4 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:20.895618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cd433-90d5-4c0a-b81e-dad939998ed4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:20.895618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cd433-90d5-4c0a-b81e-dad939998ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:20.917316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e92cd433-90d5-4c0a-b81e-dad939998ed4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:20.918533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e92cd433-90d5-4c0a-b81e-dad939998ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:21.012109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 358/721] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:21.056840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d46e5d-c6c0-4b0f-9f76-0573e1bfe29f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:21.060519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3fa7-31f6-426f-b61e-98af48ec1874 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.060811 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3fa7-31f6-426f-b61e-98af48ec1874 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.061088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3fa7-31f6-426f-b61e-98af48ec1874 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.062786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 359/722] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:21.068571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec116c62-c6ed-4121-96ca-7adb86e19c97 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.068853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec116c62-c6ed-4121-96ca-7adb86e19c97 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.069128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec116c62-c6ed-4121-96ca-7adb86e19c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.098411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ec116c62-c6ed-4121-96ca-7adb86e19c97 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:21.099593 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ec116c62-c6ed-4121-96ca-7adb86e19c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:21.122105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:21.236407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d46e5d-c6c0-4b0f-9f76-0573e1bfe29f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:21.251665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 360/723] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 22:08:21.283110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec116c62-c6ed-4121-96ca-7adb86e19c97 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:21.296072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59142568-4882-4fb2-86ad-512149fae71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:21.296767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142568-4882-4fb2-86ad-512149fae71b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.297000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142568-4882-4fb2-86ad-512149fae71b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.297271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142568-4882-4fb2-86ad-512149fae71b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.315997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59142568-4882-4fb2-86ad-512149fae71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-59142568-4882-4fb2-86ad-512149fae71b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:21.321306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:21.322405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:21.323351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 364/724] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:21.325441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09290777-595b-4de6-99a9-2b427b393e9c None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.325441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09290777-595b-4de6-99a9-2b427b393e9c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.325441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09290777-595b-4de6-99a9-2b427b393e9c None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.326859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 365/725] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:21.331717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09290777-595b-4de6-99a9-2b427b393e9c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:21.360942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:21.417139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec116c62-c6ed-4121-96ca-7adb86e19c97 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:21.439087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09290777-595b-4de6-99a9-2b427b393e9c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:21.443568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:21.468074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8f095-4e68-4bfa-82d0-a988a1897edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:21.469764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8f095-4e68-4bfa-82d0-a988a1897edb None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.469889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8f095-4e68-4bfa-82d0-a988a1897edb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.470099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8f095-4e68-4bfa-82d0-a988a1897edb None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.470842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8f095-4e68-4bfa-82d0-a988a1897edb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:21.472260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8f095-4e68-4bfa-82d0-a988a1897edb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:21.485636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:21.486799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.487096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.487527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.490048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:21.490457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:21.492446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:21.497125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 361/726] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:08:21 2022] POST /identity/v3/services => generated 252 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:08:21.498805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febea31-900c-4ba4-9ac6-4dabfe6c187e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.499291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febea31-900c-4ba4-9ac6-4dabfe6c187e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.499809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febea31-900c-4ba4-9ac6-4dabfe6c187e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.513948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 366/727] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:21 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:21.517367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:21.521180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6febea31-900c-4ba4-9ac6-4dabfe6c187e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:21.522808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6febea31-900c-4ba4-9ac6-4dabfe6c187e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:21.639796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc6d2c-a73a-423e-9c33-6c9a7080ddda None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:21.678684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 362/728] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:21 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:21.685717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:21.686519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.686804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.687083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.703288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:21.704780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:21.726000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febea31-900c-4ba4-9ac6-4dabfe6c187e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:21.727625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:21.728500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:21.729372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:21.729372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:21.729372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:21.729372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:21.729372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 367/729] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:21.734631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:21.766565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:21.782288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:21.864294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febea31-900c-4ba4-9ac6-4dabfe6c187e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:21.912525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff54cd-0cbc-4855-8160-4d0cd75ed82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:21.914374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70857e43-d08b-4720-8838-42af2e0a83af None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:21.915073 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff54cd-0cbc-4855-8160-4d0cd75ed82e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.915571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff54cd-0cbc-4855-8160-4d0cd75ed82e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.916245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff54cd-0cbc-4855-8160-4d0cd75ed82e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.935899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff54cd-0cbc-4855-8160-4d0cd75ed82e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-daff54cd-0cbc-4855-8160-4d0cd75ed82e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:21.943336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:21.944259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:21.945068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:21.946980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 363/730] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:21.954732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff54cd-0cbc-4855-8160-4d0cd75ed82e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:21.959534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:21.961131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:21.961446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:21.961729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:21.962612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:21.962921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:21.964309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:21.980944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 368/731] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:21.986393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:22.085759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:22.119776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:22.123674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff54cd-0cbc-4855-8160-4d0cd75ed82e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:22.126714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199158d-6a05-413e-b17c-4addcb0cf029 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:22.166383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:22.167430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.167809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.168180 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.177125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:22.179441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:22.193000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:22.193985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:22.194918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:22.195481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 369/732] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:22.197917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:22.198715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.198978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.199253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.199981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:22.200284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:22.201692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:22.208331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:22.216087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 364/733] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:21 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:22.222278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:22.436480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0deb0-24b6-4181-99c7-8ce15e8a3c22 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:22.474930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e961bc-7ad3-4c80-ade2-b3b7a50b8a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:22.476063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e961bc-7ad3-4c80-ade2-b3b7a50b8a29 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.476346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e961bc-7ad3-4c80-ade2-b3b7a50b8a29 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.476767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e961bc-7ad3-4c80-ade2-b3b7a50b8a29 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.477758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e961bc-7ad3-4c80-ade2-b3b7a50b8a29 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:22.478048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e961bc-7ad3-4c80-ade2-b3b7a50b8a29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:22.480123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e961bc-7ad3-4c80-ade2-b3b7a50b8a29 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:22.490580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:22.496206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4bb8-f149-4569-a115-193d35fdfd90 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:22.501606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 370/734] 10.222.0.109 () {56 vars in 1140 bytes} [Thu Aug 4 22:08:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:22.502946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8c730-967e-4dd0-9292-05066d8e62ad None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.503356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8c730-967e-4dd0-9292-05066d8e62ad None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.503655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8c730-967e-4dd0-9292-05066d8e62ad None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.508802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 371/735] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:22.508802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64359ec6-e41c-4fb3-a5d1-1f625f4cd58d None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.508802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64359ec6-e41c-4fb3-a5d1-1f625f4cd58d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.508802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64359ec6-e41c-4fb3-a5d1-1f625f4cd58d None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.508802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 372/736] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:22.509216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64359ec6-e41c-4fb3-a5d1-1f625f4cd58d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:22.539265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca5820-1a4a-4ac2-b4a7-6135d54aa92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:22.540020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca5820-1a4a-4ac2-b4a7-6135d54aa92e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.540299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca5820-1a4a-4ac2-b4a7-6135d54aa92e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.540602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca5820-1a4a-4ac2-b4a7-6135d54aa92e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.552543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca5820-1a4a-4ac2-b4a7-6135d54aa92e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:22.554004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca5820-1a4a-4ac2-b4a7-6135d54aa92e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-93ca5820-1a4a-4ac2-b4a7-6135d54aa92e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:22.568938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:22.570050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:22.571020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:22.571020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:22.571020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:22.571020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:22.571020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 365/737] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:22.571315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca4f4-ed30-4fa8-b54c-a4baeaf4ba89 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.571755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca4f4-ed30-4fa8-b54c-a4baeaf4ba89 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.571860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca4f4-ed30-4fa8-b54c-a4baeaf4ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.591456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-26aca4f4-ed30-4fa8-b54c-a4baeaf4ba89 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:22.592706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-26aca4f4-ed30-4fa8-b54c-a4baeaf4ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:22.601307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:22.625620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64359ec6-e41c-4fb3-a5d1-1f625f4cd58d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:22.665029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386e13a-5efc-4359-80e0-707846742793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:22.665805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386e13a-5efc-4359-80e0-707846742793 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.666086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386e13a-5efc-4359-80e0-707846742793 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.666359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386e13a-5efc-4359-80e0-707846742793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.669397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 366/738] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:22 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:22.670866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3b16f-10bd-4274-8b47-afbade5a88dc None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.671221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3b16f-10bd-4274-8b47-afbade5a88dc None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.671544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3b16f-10bd-4274-8b47-afbade5a88dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.692612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-dcf3b16f-10bd-4274-8b47-afbade5a88dc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:22.693856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-dcf3b16f-10bd-4274-8b47-afbade5a88dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:22.741675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:22.750168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386e13a-5efc-4359-80e0-707846742793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:22.751626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386e13a-5efc-4359-80e0-707846742793 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:22.773757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 367/739] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:22 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 22:08:22.775472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3b16f-10bd-4274-8b47-afbade5a88dc None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:22.854762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:22.864376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 373/740] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:22 2022] GET /identity/v3/auth/tokens => generated 904 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 22:08:22.865072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386e13a-5efc-4359-80e0-707846742793 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:22.888346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3b16f-10bd-4274-8b47-afbade5a88dc None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:22.926234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:22.927231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:22.927639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:22.928042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:22.928723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:22.929174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:22.931165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:22.945649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 368/741] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:22.946435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:22.970441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:22.980536 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386e13a-5efc-4359-80e0-707846742793 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.026818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc9162-f02b-49fa-93a7-fcf4e5c95de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.027467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc9162-f02b-49fa-93a7-fcf4e5c95de8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.027747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc9162-f02b-49fa-93a7-fcf4e5c95de8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.028020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc9162-f02b-49fa-93a7-fcf4e5c95de8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.038903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc9162-f02b-49fa-93a7-fcf4e5c95de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-41fc9162-f02b-49fa-93a7-fcf4e5c95de8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:23.043870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:23.044777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:23.045636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 374/742] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:23.046773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc9162-f02b-49fa-93a7-fcf4e5c95de8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.064429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ac0e-c3b6-4519-bbaa-3397cc274201 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.096409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:23.105657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a317780-9deb-41d2-87b2-6021801f7d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.108459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a317780-9deb-41d2-87b2-6021801f7d4f None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.108459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a317780-9deb-41d2-87b2-6021801f7d4f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.108459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a317780-9deb-41d2-87b2-6021801f7d4f None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.108459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a317780-9deb-41d2-87b2-6021801f7d4f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.108998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a317780-9deb-41d2-87b2-6021801f7d4f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.125453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 369/743] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:08:22 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:23.125453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a317780-9deb-41d2-87b2-6021801f7d4f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.191950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc9162-f02b-49fa-93a7-fcf4e5c95de8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.237459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.238444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.238934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.239390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.244700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a317780-9deb-41d2-87b2-6021801f7d4f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.251535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.253350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:23.272135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:23.274008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:23.275836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:23.275836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:23.275836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:23.275836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:23.283592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 375/744] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:23.284774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.287053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.287755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.288039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.288310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] PATH_INFO: `/v3/users/nova` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.290695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:23.297276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.298789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:23.311000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:23.311968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 370/745] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:23.313378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.395748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:23.412410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0168b-3e63-4601-87d4-4eecaa8feae7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.450815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.451252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f900ed-a5e1-4096-8798-1cad790f924d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.451426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.451711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.451985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.452702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:23.453006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.454541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.464297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 376/746] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:23.465035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.490084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.491164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.491642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.492122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.493006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:23.493642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.495218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.509509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 371/747] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:23.510829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.542807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:23.584316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbe902-f122-4e63-a35a-0f03777816f6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.621297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.621971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.622376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.622626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.623684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:23.623803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.625247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.630922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820850ad-8be1-41d6-805e-9794d1af2ca6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.647766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 377/748] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:23.647766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.654784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:23.680699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.681639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.682017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.682380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.695314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.697083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:23.715926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:23.717122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:23.718255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:23.718255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:23.718255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:23.718255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:23.724102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 372/749] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:23.724102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.772128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d3e77-eda1-40f7-8b70-477954071b6a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.804107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:23.820088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.821080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.821543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.821965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.829783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.832380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.837757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa383d52-2246-4caf-86fa-235b6b914849 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:23.846585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:23.847997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:23.849266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:23.849266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:23.849266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:23.849266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:23.849266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:23.849266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:23.849266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:23.849266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:23.851505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 378/750] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:23.852606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.876300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:23.877328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:23.877991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:23.879099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:23.890781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:23.892364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:23.908429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:23.909825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:23.910994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:23.910994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:23.910994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:23.910994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:23.916648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 373/751] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:23.917835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:23.937354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:23.990947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d726d-a090-4b70-8c7a-d0d9bf89287b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.030867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.031411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.031701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.031976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.032667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:24.032974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.033190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ed98c-9c25-44d4-b752-2e1558b0f48d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.034390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.048996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 379/752] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:24.050552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.075035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923de68e-39d9-4504-91ae-76bbfe4331d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.075814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923de68e-39d9-4504-91ae-76bbfe4331d3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.076099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923de68e-39d9-4504-91ae-76bbfe4331d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.076371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923de68e-39d9-4504-91ae-76bbfe4331d3 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.077185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923de68e-39d9-4504-91ae-76bbfe4331d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:24.077606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923de68e-39d9-4504-91ae-76bbfe4331d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.079540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923de68e-39d9-4504-91ae-76bbfe4331d3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.097755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 374/753] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:24.100012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333fc9b-4dd5-44f8-85a3-410de5df488d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.100521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333fc9b-4dd5-44f8-85a3-410de5df488d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.101014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333fc9b-4dd5-44f8-85a3-410de5df488d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.102568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 375/754] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:24.103471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333fc9b-4dd5-44f8-85a3-410de5df488d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.114476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:24.174872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e21ea-737f-445d-bfa4-6429f6b58dd6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.196725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:24.213998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.214907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.215248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.215599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.216316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:24.216959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.218377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.226575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333fc9b-4dd5-44f8-85a3-410de5df488d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.236274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 380/755] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:24.236274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.269363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb5497-20c2-49ea-99c3-131dd660d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.270192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb5497-20c2-49ea-99c3-131dd660d8bd None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.270557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb5497-20c2-49ea-99c3-131dd660d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.270836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb5497-20c2-49ea-99c3-131dd660d8bd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.279253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb5497-20c2-49ea-99c3-131dd660d8bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.281144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb5497-20c2-49ea-99c3-131dd660d8bd None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-7fcb5497-20c2-49ea-99c3-131dd660d8bd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:24.296758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:24.297802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:24.298734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:24.299363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 376/756] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:24.301544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac8794-80ec-4cda-8ede-bf404639b6b9 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.302004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac8794-80ec-4cda-8ede-bf404639b6b9 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.302521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac8794-80ec-4cda-8ede-bf404639b6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.321633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-26ac8794-80ec-4cda-8ede-bf404639b6b9 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:24.322842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-26ac8794-80ec-4cda-8ede-bf404639b6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:24.360092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a83d4-1c23-44f5-bb98-177593e55be9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.363592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:24.403560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.404194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.404477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.404755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.406289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=848151a99fb74f6b8f8210cfc49afeff) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:24.406613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.413468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 377/757] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:24 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:24.413468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac8794-80ec-4cda-8ede-bf404639b6b9 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.413728 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.429884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 381/758] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/role_assignments?user.id=848151a99fb74f6b8f8210cfc49afeff&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:24.431016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.504384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:24.547137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac8794-80ec-4cda-8ede-bf404639b6b9 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.584541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005061d2-ad99-4c0d-8c44-d233c8c91aa6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.588351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.589054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.589396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.589766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.598026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.599647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:24.611564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:24.612473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:24.614001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 378/759] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:24.614489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.633186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.634121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.634585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.635045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.636048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:24.636537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.638179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.662114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 382/760] 10.222.0.109 () {56 vars in 1154 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:24.663494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.693036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:24.723048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf1790-2748-42fe-8645-f2bc2fd0da0d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.750403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:24.760589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.761237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.761599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.761890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.762531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.763915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.773531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213d2e2-92a0-4790-9327-4909c74f9d90 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:24.773855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:24.774768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:24.775987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 379/761] 10.222.0.109 () {56 vars in 1134 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:24.776849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.820089 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.821077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.821722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.822207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.823174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:24.823687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.825360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.838660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 383/762] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:24.839980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:24.891854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119273fa-e521-4ede-9161-f1af3d0caffd None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.928722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff3a0f-5fc5-43bd-a48a-49815b1f71e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.929650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff3a0f-5fc5-43bd-a48a-49815b1f71e2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.929767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff3a0f-5fc5-43bd-a48a-49815b1f71e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.930001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff3a0f-5fc5-43bd-a48a-49815b1f71e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.933423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:24.949957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105c3fe-6434-47d1-946b-eb2ea5377091 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:24.986756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:24.987740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:24.988395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:24.988898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:24.989823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:24.990359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:24.991951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:24.996789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:24.998471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 384/763] 10.222.0.109 () {56 vars in 1129 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:24.999818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.014591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff3a0f-5fc5-43bd-a48a-49815b1f71e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:25.015972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff3a0f-5fc5-43bd-a48a-49815b1f71e2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:25.119161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db1018-e098-41e4-abfa-21df5a620d90 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:25.140974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 380/764] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:24 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:08:25.142232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff3a0f-5fc5-43bd-a48a-49815b1f71e2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.163520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e72c4-5212-49a3-9a45-9e42e4103e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:25.164560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e72c4-5212-49a3-9a45-9e42e4103e66 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.165063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e72c4-5212-49a3-9a45-9e42e4103e66 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.165614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e72c4-5212-49a3-9a45-9e42e4103e66 None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/41d9317e24714de1a310198bb35eb00b/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.167846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e72c4-5212-49a3-9a45-9e42e4103e66 None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=41d9317e24714de1a310198bb35eb00b, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:25.169608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e72c4-5212-49a3-9a45-9e42e4103e66 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:25.186735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 385/765] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:24 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/41d9317e24714de1a310198bb35eb00b/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:25.187970 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e72c4-5212-49a3-9a45-9e42e4103e66 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.221809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:25.325814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:25.330337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff3a0f-5fc5-43bd-a48a-49815b1f71e2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:25.388361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:25.389740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.390225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.390696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.391698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:25.392204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:25.394092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:25.403394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 381/766] 10.222.0.109 () {56 vars in 1129 bytes} [Thu Aug 4 22:08:25 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:25.405318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e72c4-5212-49a3-9a45-9e42e4103e66 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:25.406925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.444651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe519ae9-f5e3-4671-a4f3-5a2c66dc1121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:25.445272 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519ae9-f5e3-4671-a4f3-5a2c66dc1121 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.445603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519ae9-f5e3-4671-a4f3-5a2c66dc1121 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.445878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519ae9-f5e3-4671-a4f3-5a2c66dc1121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.460676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519ae9-f5e3-4671-a4f3-5a2c66dc1121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:25.462242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe519ae9-f5e3-4671-a4f3-5a2c66dc1121 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-fe519ae9-f5e3-4671-a4f3-5a2c66dc1121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:25.477846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:25.478867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:25.479771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:25.479771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:25.479771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:25.479771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:25.482590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 386/767] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:25.483844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5783d-3b31-4d4a-9e84-dab03b266111 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.484146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5783d-3b31-4d4a-9e84-dab03b266111 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.484472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5783d-3b31-4d4a-9e84-dab03b266111 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.486979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 387/768] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:25.488736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5783d-3b31-4d4a-9e84-dab03b266111 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.528649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ff473-cb0e-4f3c-923e-d9791af3c739 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:25.562603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:25.568350 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c829833-6c5e-4f88-ba86-36cc64f47099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:25.569299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c829833-6c5e-4f88-ba86-36cc64f47099 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.569615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c829833-6c5e-4f88-ba86-36cc64f47099 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.569896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c829833-6c5e-4f88-ba86-36cc64f47099 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.570661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c829833-6c5e-4f88-ba86-36cc64f47099 None admin] RBAC: Adding query filter params (interface=public, service_id=cc5d6ec4623a4a7296f3dd6e513c6ab7, region_id=RegionOne) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:25.570971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c829833-6c5e-4f88-ba86-36cc64f47099 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:25.572356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c829833-6c5e-4f88-ba86-36cc64f47099 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:25.580737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 382/769] 10.222.0.109 () {56 vars in 1256 bytes} [Thu Aug 4 22:08:25 2022] GET /identity/v3/endpoints?service_id=cc5d6ec4623a4a7296f3dd6e513c6ab7&interface=public®ion_id=RegionOne => generated 195 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:25.582878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738ec93-7747-47ee-8a75-d85aa2cfdaac None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.583055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738ec93-7747-47ee-8a75-d85aa2cfdaac None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.583238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738ec93-7747-47ee-8a75-d85aa2cfdaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.602526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7738ec93-7747-47ee-8a75-d85aa2cfdaac None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:25.603755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7738ec93-7747-47ee-8a75-d85aa2cfdaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:25.610090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5783d-3b31-4d4a-9e84-dab03b266111 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:25.619262 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:25.647603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:25.648142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.648423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.648693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.649386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:25.649747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:25.651206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:25.669202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 388/770] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:25.670156 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.681076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 383/771] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:25 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:25.682260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738ec93-7747-47ee-8a75-d85aa2cfdaac None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.780386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36527c-3a19-4e3c-b60d-77363ec5e32f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:25.785272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738ec93-7747-47ee-8a75-d85aa2cfdaac None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:25.806095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:25.821263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51843dc8-0f36-4d6d-827d-43736cbb3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:25.821926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843dc8-0f36-4d6d-827d-43736cbb3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.822214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843dc8-0f36-4d6d-827d-43736cbb3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.822495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843dc8-0f36-4d6d-827d-43736cbb3ae6 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.823165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51843dc8-0f36-4d6d-827d-43736cbb3ae6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:25.824663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51843dc8-0f36-4d6d-827d-43736cbb3ae6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:25.826264 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:25.827109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:25.827588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:25.828051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:25.830450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 389/772] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:08:25 2022] GET /identity/v3/services => generated 776 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:25.831676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51843dc8-0f36-4d6d-827d-43736cbb3ae6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.836510 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:25.838365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:25.853741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:25.854733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:25.855627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:25.855627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:25.855627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:25.855627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:25.855627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:25.855627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:25.855627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:25.855627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:25.858414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 384/773] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:25.865456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:25.878061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:25.971240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51843dc8-0f36-4d6d-827d-43736cbb3ae6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:25.990730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7168e-184b-43ea-b182-bc75128571ea None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:26.020893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:26.021760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.022046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.022322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] PATH_INFO: `/v3/services/network` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.023416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:26.025121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:26.031804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:26.032976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.033473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.033992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.035543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:26.036032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:26.037671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:26.037932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:26.038899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:26.039723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 390/774] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:25 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:26.048736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:26.059054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 385/775] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:26.065374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:26.145680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:26.154073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:26.189813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35157f-6bb3-4c98-82c0-1bdec31d59d3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:26.222615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87e960-b038-4bef-b812-331ab6cbbb21 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:26.229363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:26.230278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.230753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.231265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.232236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=212a75e45f2b449398acbfb80338afb0) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:26.232730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:26.235335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:26.242877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 386/776] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:26 2022] GET /identity/v3/role_assignments?user.id=212a75e45f2b449398acbfb80338afb0&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:26.267484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:26.268522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.268869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.270275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.271131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] RBAC: Adding query filter params (name=network) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:26.271502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:26.272999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:26.278740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 391/777] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:26 2022] GET /identity/v3/services?name=network => generated 125 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:26.284090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:26.407131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e40b-532b-4866-837f-924a22455a5a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:26.440937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:26.449768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d7c21-6cae-4c45-b41f-a5df955dd283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:26.450884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d7c21-6cae-4c45-b41f-a5df955dd283 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.451354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d7c21-6cae-4c45-b41f-a5df955dd283 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.451628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d7c21-6cae-4c45-b41f-a5df955dd283 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.452388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d7c21-6cae-4c45-b41f-a5df955dd283 None admin] RBAC: Adding query filter params (type=network) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:26.452695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d7c21-6cae-4c45-b41f-a5df955dd283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:26.454106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d7c21-6cae-4c45-b41f-a5df955dd283 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:26.461050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 387/778] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:26 2022] GET /identity/v3/services?type=network => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:26.466704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:26.587615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fa30b-d1c9-4463-9137-5212521b5ea1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:26.604762 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:26.635185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:26.635817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.636099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.636354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.637104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] RBAC: Adding query filter params (interface=public, service_id=124c4c4511a646c68359b6941fa63735, region_id=RegionOne) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:26.637424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:26.638790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:26.651897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 392/779] 10.222.0.109 () {56 vars in 1256 bytes} [Thu Aug 4 22:08:26 2022] GET /identity/v3/endpoints?service_id=124c4c4511a646c68359b6941fa63735&interface=public®ion_id=RegionOne => generated 195 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:26.654352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7183ca-2b9a-4b63-af25-d81fbb6756bc None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.654638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7183ca-2b9a-4b63-af25-d81fbb6756bc None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.654915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7183ca-2b9a-4b63-af25-d81fbb6756bc None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.656479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 388/780] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:26.656992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:26.661947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263163b4-6784-4816-af67-864a6282bdba None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.662227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263163b4-6784-4816-af67-864a6282bdba None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.662502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263163b4-6784-4816-af67-864a6282bdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.693114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-263163b4-6784-4816-af67-864a6282bdba None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:26.694352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-263163b4-6784-4816-af67-864a6282bdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:26.768394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315150c-5429-4f3d-8b18-2d681ee24306 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:26.818816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b574d1f-d892-418f-be9e-a0cdd2d825e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:26.819505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574d1f-d892-418f-be9e-a0cdd2d825e1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:26.819789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574d1f-d892-418f-be9e-a0cdd2d825e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:26.820064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574d1f-d892-418f-be9e-a0cdd2d825e1 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:26.820729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b574d1f-d892-418f-be9e-a0cdd2d825e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:26.822207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b574d1f-d892-418f-be9e-a0cdd2d825e1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:26.829511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 389/781] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:26 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:26.835188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 393/782] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:08:26 2022] GET /identity/v3/services => generated 776 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:26.854125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263163b4-6784-4816-af67-864a6282bdba None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:26.999238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263163b4-6784-4816-af67-864a6282bdba None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:27.041518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:27.051099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d9ea2-f781-4d3e-b5a1-c11212509662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:27.051738 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9ea2-f781-4d3e-b5a1-c11212509662 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:27.052012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9ea2-f781-4d3e-b5a1-c11212509662 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:27.052278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9ea2-f781-4d3e-b5a1-c11212509662 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:27.063246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d9ea2-f781-4d3e-b5a1-c11212509662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-384d9ea2-f781-4d3e-b5a1-c11212509662 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:27.068803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:27.069844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:27.071147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 390/783] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:27.076080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b574d1f-d892-418f-be9e-a0cdd2d825e1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:27.134470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:27.195076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b574d1f-d892-418f-be9e-a0cdd2d825e1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:27.239334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:27.240298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:27.240753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:27.241195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:27.242111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:27.242587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:27.244147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:27.253321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 394/784] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:27.259326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d9ea2-f781-4d3e-b5a1-c11212509662 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:27.395942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d9ea2-f781-4d3e-b5a1-c11212509662 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:27.455227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:27.455888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:27.456168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:27.456515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:27.474683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:27.476330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:27.495535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:27.496448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:27.497321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:27.497321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:27.497321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:27.497321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:27.497321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 391/785] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:27.502495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:27.634352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bfd50-863d-4933-ab24-70f81d571d87 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:27.637102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:27.671823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686c58-1efe-416a-9bf2-539567b4eafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:27.672785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686c58-1efe-416a-9bf2-539567b4eafb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:27.673259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686c58-1efe-416a-9bf2-539567b4eafb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:27.673810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686c58-1efe-416a-9bf2-539567b4eafb None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:27.674720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40686c58-1efe-416a-9bf2-539567b4eafb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:27.675872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40686c58-1efe-416a-9bf2-539567b4eafb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:27.677606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40686c58-1efe-416a-9bf2-539567b4eafb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:27.690184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 395/786] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:27.697535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:27.761202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:27.827994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa350a-e97f-45ab-b0f4-a216742ed7f7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:27.865443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:27.866183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:27.866411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:27.866744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:27.868517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7fce1-b0f4-4594-9750-250e3267bdca None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:27.868813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7fce1-b0f4-4594-9750-250e3267bdca None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:27.869089 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7fce1-b0f4-4594-9750-250e3267bdca None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:27.871496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 396/787] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:27.874654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:27.875680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076c5b-3e49-4cf0-b3b0-3e55133f0641 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:27.875973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076c5b-3e49-4cf0-b3b0-3e55133f0641 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:27.876252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076c5b-3e49-4cf0-b3b0-3e55133f0641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:27.876356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:27.889108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:27.890097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:27.891048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 392/788] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:27.896971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:27.921263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-88076c5b-3e49-4cf0-b3b0-3e55133f0641 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:27.922556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-88076c5b-3e49-4cf0-b3b0-3e55133f0641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:28.054011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 397/789] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:27 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:28.072312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3da1-7539-4faa-ad11-94910c25e543 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:28.082307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:28.089016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076c5b-3e49-4cf0-b3b0-3e55133f0641 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:28.126847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:28.127483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.127762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.128035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.129792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:28.130102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:28.131538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:28.173734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 393/790] 10.222.0.109 () {56 vars in 1140 bytes} [Thu Aug 4 22:08:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:28.178595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:28.270023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:28.284618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076c5b-3e49-4cf0-b3b0-3e55133f0641 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:28.351503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33bde5-3ec5-4cd2-ae12-827ee5f32e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:28.352165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33bde5-3ec5-4cd2-ae12-827ee5f32e6d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.352444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33bde5-3ec5-4cd2-ae12-827ee5f32e6d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.352718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33bde5-3ec5-4cd2-ae12-827ee5f32e6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.374008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33bde5-3ec5-4cd2-ae12-827ee5f32e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:28.379882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e88a8-048d-46ae-8c71-f5c45eaeb65c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-3f33bde5-3ec5-4cd2-ae12-827ee5f32e6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:28.384458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:28.385508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:28.386335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 398/791] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:28.394619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33bde5-3ec5-4cd2-ae12-827ee5f32e6d None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:28.463902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:28.464904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.465382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.465978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.481444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:28.483468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:28.502030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:28.502962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:28.503914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:28.503914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:28.503914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:28.503914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:28.506725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 394/792] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:28.516686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:28.538180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:28.602826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33bde5-3ec5-4cd2-ae12-827ee5f32e6d None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:28.665290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:28.665954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.666307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.666590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.667314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:28.667622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:28.669100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:28.680091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 399/793] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:28 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:28.687577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:28.713305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fd96-6c71-47f6-a82e-88ceb7abaa3a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:28.749650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738271d-ce02-4a29-9c0a-8ef318613ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:28.750199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738271d-ce02-4a29-9c0a-8ef318613ab7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.750537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738271d-ce02-4a29-9c0a-8ef318613ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.750812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738271d-ce02-4a29-9c0a-8ef318613ab7 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.751560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738271d-ce02-4a29-9c0a-8ef318613ab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:28.751870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738271d-ce02-4a29-9c0a-8ef318613ab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:28.752402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:28.753457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738271d-ce02-4a29-9c0a-8ef318613ab7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:28.766769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 395/794] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:28.770902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3230e4-28ce-4f9b-a134-2dbb43713346 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.771420 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3230e4-28ce-4f9b-a134-2dbb43713346 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.771725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3230e4-28ce-4f9b-a134-2dbb43713346 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.773564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 396/795] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:28 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:28.774729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7b234-661e-4f0f-b19b-aaae6d642903 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.775020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7b234-661e-4f0f-b19b-aaae6d642903 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.775295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7b234-661e-4f0f-b19b-aaae6d642903 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.776785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 397/796] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:28.777525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7b234-661e-4f0f-b19b-aaae6d642903 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:28.824346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac8327-abd5-4aa3-a9a9-54cbb17dfef9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:28.851740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:28.861579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107546a-f8af-4aac-9cce-65b6a28fe33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:28.862174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107546a-f8af-4aac-9cce-65b6a28fe33b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.862457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107546a-f8af-4aac-9cce-65b6a28fe33b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.862842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107546a-f8af-4aac-9cce-65b6a28fe33b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.874179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107546a-f8af-4aac-9cce-65b6a28fe33b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:28.875717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107546a-f8af-4aac-9cce-65b6a28fe33b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:28.888821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7b234-661e-4f0f-b19b-aaae6d642903 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-b107546a-f8af-4aac-9cce-65b6a28fe33b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:28.891109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:28.892002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:28.893152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:28.893152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:28.893152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:28.893152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:28.897757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 400/797] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:28.898928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6c66-9edd-48a7-acc0-283634fceca0 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.899221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6c66-9edd-48a7-acc0-283634fceca0 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.899500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6c66-9edd-48a7-acc0-283634fceca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.917966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-123a6c66-9edd-48a7-acc0-283634fceca0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:28.919167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-123a6c66-9edd-48a7-acc0-283634fceca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:28.930480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dbf16-575d-4ce2-9e44-fcadd27267c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:28.931271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbf16-575d-4ce2-9e44-fcadd27267c6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.931383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbf16-575d-4ce2-9e44-fcadd27267c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.931656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbf16-575d-4ce2-9e44-fcadd27267c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.938639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dbf16-575d-4ce2-9e44-fcadd27267c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:28.940077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dbf16-575d-4ce2-9e44-fcadd27267c6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-8a0dbf16-575d-4ce2-9e44-fcadd27267c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:28.952726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:28.953678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:28.954609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:28.954609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:28.954609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:28.954609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:28.954609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:28.954609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:28.954609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:28.954609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:28.958399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 398/798] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:28.959870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ebbe1-478b-4f58-8aaf-1dbafe6739ae None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:28.960157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ebbe1-478b-4f58-8aaf-1dbafe6739ae None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:28.960433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ebbe1-478b-4f58-8aaf-1dbafe6739ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:28.979903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-455ebbe1-478b-4f58-8aaf-1dbafe6739ae None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:28.981100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-455ebbe1-478b-4f58-8aaf-1dbafe6739ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:28.987056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:28.997512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 401/799] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:28 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:28.997512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a6c66-9edd-48a7-acc0-283634fceca0 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.055809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 399/800] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:28 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:29.056572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ebbe1-478b-4f58-8aaf-1dbafe6739ae None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.116696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a6c66-9edd-48a7-acc0-283634fceca0 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.128112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:29.152405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.153096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.153377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.153715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.154418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:29.154723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:29.156107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:29.168035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 402/801] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:29.168862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.182553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ebbe1-478b-4f58-8aaf-1dbafe6739ae None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.218719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.219333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.219618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.219886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.220631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:29.220936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:29.223439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:29.232643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:29.236223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 400/802] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:29.242440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.279817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf1d83-a6bf-422f-8c4d-51f75e1a60cf None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.319006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc3797-d7cd-4070-b3f7-11e4ba485177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.319573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3797-d7cd-4070-b3f7-11e4ba485177 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.319854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3797-d7cd-4070-b3f7-11e4ba485177 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.320123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3797-d7cd-4070-b3f7-11e4ba485177 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.333446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc3797-d7cd-4070-b3f7-11e4ba485177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-b7cc3797-d7cd-4070-b3f7-11e4ba485177 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:29.338883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:29.339860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:29.340869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 403/803] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:29.341824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc3797-d7cd-4070-b3f7-11e4ba485177 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.361318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc19914-950e-40c5-876b-fc457af200e7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.379803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:29.396588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.397235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.397547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.397826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.398453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:29.399919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:29.408736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:29.409932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 22:08:29.410864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:29.413546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 401/804] 10.222.0.109 () {56 vars in 1134 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:29.414466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.448509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc3797-d7cd-4070-b3f7-11e4ba485177 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.466328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:29.485244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df1c36-1882-4550-ae9e-6c45c54c0802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.486018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df1c36-1882-4550-ae9e-6c45c54c0802 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.486306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df1c36-1882-4550-ae9e-6c45c54c0802 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.486581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df1c36-1882-4550-ae9e-6c45c54c0802 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.495640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df1c36-1882-4550-ae9e-6c45c54c0802 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:29.497642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df1c36-1882-4550-ae9e-6c45c54c0802 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-b9df1c36-1882-4550-ae9e-6c45c54c0802 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:29.515677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:29.516820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:29.518231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 404/805] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:29.519890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4e5d5-3842-4372-8a93-135ddecd22c0 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.520307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4e5d5-3842-4372-8a93-135ddecd22c0 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.520714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4e5d5-3842-4372-8a93-135ddecd22c0 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.522916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 405/806] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:29 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:29.523975 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4e5d5-3842-4372-8a93-135ddecd22c0 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.528193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386eed7-f70e-4169-9164-02e263b4a20d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.563472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038feca-ccc7-41b0-a719-3824756ba5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.564134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038feca-ccc7-41b0-a719-3824756ba5f7 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.564406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038feca-ccc7-41b0-a719-3824756ba5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.564679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038feca-ccc7-41b0-a719-3824756ba5f7 None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/848151a99fb74f6b8f8210cfc49afeff/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.567168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038feca-ccc7-41b0-a719-3824756ba5f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=848151a99fb74f6b8f8210cfc49afeff, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:29.568586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038feca-ccc7-41b0-a719-3824756ba5f7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:29.580726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 402/807] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:29 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/848151a99fb74f6b8f8210cfc49afeff/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:29.581587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038feca-ccc7-41b0-a719-3824756ba5f7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.644598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4e5d5-3842-4372-8a93-135ddecd22c0 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.649631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:29.683378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.684070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.684356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.684633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.685359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:29.685695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:29.687102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:29.698300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 406/808] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:29.699175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.731721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038feca-ccc7-41b0-a719-3824756ba5f7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.746403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:29.777892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28e238-b3f7-4fb1-844e-7639fcdb1e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.778508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28e238-b3f7-4fb1-844e-7639fcdb1e05 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.778784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28e238-b3f7-4fb1-844e-7639fcdb1e05 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.779052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28e238-b3f7-4fb1-844e-7639fcdb1e05 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.779719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28e238-b3f7-4fb1-844e-7639fcdb1e05 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:29.780015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28e238-b3f7-4fb1-844e-7639fcdb1e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:29.781373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28e238-b3f7-4fb1-844e-7639fcdb1e05 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:29.786462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 403/809] 10.222.0.109 () {56 vars in 1129 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:29.787637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5d0e-760f-4a68-aa79-703b7804cd5e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5d0e-760f-4a68-aa79-703b7804cd5e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.788189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5d0e-760f-4a68-aa79-703b7804cd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.806277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-c2cd5d0e-760f-4a68-aa79-703b7804cd5e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:29.807511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c2cd5d0e-760f-4a68-aa79-703b7804cd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:29.810385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b8513-1a56-4504-bcc7-216c30e2d8a8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:29.852434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:29.852966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:29.853242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:29.853546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:29.854342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:29.854645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:29.856068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:29.878965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 407/810] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:29.878965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.888351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 404/811] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:29 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:29.897490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd5d0e-760f-4a68-aa79-703b7804cd5e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:29.934926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:29.993701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:29.998386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce627887-2e2d-467e-8fbb-e9f27200ba67 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.002496 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd5d0e-760f-4a68-aa79-703b7804cd5e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.035511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.036154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.036468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.036694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.038609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.039327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.039704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.040073 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.040878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:30.041275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.042761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.047532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.052635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 405/812] 10.222.0.109 () {56 vars in 1129 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:30.052635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.052900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:30.070562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:30.071686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:30.072605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:30.072605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:30.072605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:30.072605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:30.072605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 408/813] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:30.072879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.171789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:30.176218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef64855-c626-4a24-8249-4ecac2a2d957 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.183770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6b76e-6805-4ac5-8a5f-e4f8bd385a48 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.212986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f0d5e-add2-4fb5-b9f6-c249605c7d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.213915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0d5e-add2-4fb5-b9f6-c249605c7d1c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.214355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0d5e-add2-4fb5-b9f6-c249605c7d1c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.214628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0d5e-add2-4fb5-b9f6-c249605c7d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.220992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.221759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.222155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.223155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] PATH_INFO: `/v3/services/network` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.223788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.225450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:30.234965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:30.235938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:30.236791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 409/814] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:30.238653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.255602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:30.294298 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f0d5e-add2-4fb5-b9f6-c249605c7d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.295746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f0d5e-add2-4fb5-b9f6-c249605c7d1c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.344623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6e301-63d3-42ba-aac0-0e45e8cd2e06 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.381272 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13fb65-bc30-4f8d-82df-3b44e8b32cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.381925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13fb65-bc30-4f8d-82df-3b44e8b32cea None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.382208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13fb65-bc30-4f8d-82df-3b44e8b32cea None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.382478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13fb65-bc30-4f8d-82df-3b44e8b32cea None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.383181 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13fb65-bc30-4f8d-82df-3b44e8b32cea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.384549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13fb65-bc30-4f8d-82df-3b44e8b32cea None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.407441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 410/815] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:08:30 2022] POST /identity/v3/endpoints => generated 348 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:30.408246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13fb65-bc30-4f8d-82df-3b44e8b32cea None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.419939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:30.425597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 406/816] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:08:30.426596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f0d5e-add2-4fb5-b9f6-c249605c7d1c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.501482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:30.549999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13fb65-bc30-4f8d-82df-3b44e8b32cea None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.578428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f0d5e-add2-4fb5-b9f6-c249605c7d1c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.589146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.590082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.590082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.590349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.591017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:30.591314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.593493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.607642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 411/817] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:30.608558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.624364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.625216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.625685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.625958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.627180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] RBAC: Adding query filter params (name=network) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:30.627180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.628569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.637772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 407/818] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/services?name=network => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:30.637772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.732629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:30.732828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784bcfd-c44a-45f2-9e94-9490f1dff73d None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.763870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:30.774950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.775693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.776043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.776438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.786087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ab7e2-1bb3-42ee-894c-c462718ec3e1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.788629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.790604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:30.813038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:30.814051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:30.814937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:30.814937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:30.814937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:30.814937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:30.821630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 412/819] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:30.822725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.829218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.830002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.830404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.830751 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.839223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.841046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:30.854708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:30.855668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:30.856674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:30.860943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 408/820] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:30.861927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:30.935675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e77859-328f-4225-bac6-94df5af7233b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.970317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d8cb0-54a9-43cc-ad88-93b1102971cf None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:30.972223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:30.973094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:30.973617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:30.974110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:30.974981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] RBAC: Adding query filter params (type=network) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:30.975476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:30.977047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:30.988585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 413/821] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/services?type=network => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:30.989846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.008348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.008978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.009315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.009680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.010390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:31.010699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.012168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.013762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:31.030260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 409/822] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:31.030260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.044826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:31.113805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a426f-811a-44a2-9102-54aacc52034f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:31.150195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.151167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.151668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.152135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.153135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:31.153668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.155306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.162549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d94fe3-304e-4899-873b-08ce615df9b3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:31.175614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 414/823] 10.222.0.109 () {56 vars in 1154 bytes} [Thu Aug 4 22:08:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:31.176926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.207255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2455a2-4909-41a7-9c00-7a1608094bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.207772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2455a2-4909-41a7-9c00-7a1608094bd6 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.208002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2455a2-4909-41a7-9c00-7a1608094bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.208274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2455a2-4909-41a7-9c00-7a1608094bd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.209014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2455a2-4909-41a7-9c00-7a1608094bd6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.210450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2455a2-4909-41a7-9c00-7a1608094bd6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.245438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 410/824] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:08:31 2022] POST /identity/v3/endpoints => generated 339 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:31.245438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2455a2-4909-41a7-9c00-7a1608094bd6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.343372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:31.364923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:31.368963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4d947-6039-41d9-81f6-6dab75aa638a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:31.433137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2455a2-4909-41a7-9c00-7a1608094bd6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:31.449703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.450748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.451223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.451700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.460843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.462655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.475521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.476224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.476511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.476785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:31.477794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:31.478642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:31.479513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:31.479513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:31.479513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:31.479513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:31.479513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:31.479513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:31.479513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:31.479513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:31.481848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 415/825] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:31.488103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.489998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.491624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:31.508007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:31.508928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:31.509817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:31.509817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:31.509817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:31.509817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:31.513790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 411/826] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:31.515275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.598727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d5b81-5c7f-4dca-9a28-db7377296f2e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:31.621121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:31.635058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.635659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.636044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.636407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.637247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:31.637689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.638949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c5045-8bcd-4448-adb8-0c007c9ba4bf None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:31.639312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.653610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 416/827] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:31.660245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.682325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a830f36-0db9-4124-b8ec-f597226011ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.682956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a830f36-0db9-4124-b8ec-f597226011ad None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.683237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a830f36-0db9-4124-b8ec-f597226011ad None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.683798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:31.684370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a830f36-0db9-4124-b8ec-f597226011ad None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.684906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a830f36-0db9-4124-b8ec-f597226011ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:31.685441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a830f36-0db9-4124-b8ec-f597226011ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.686817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a830f36-0db9-4124-b8ec-f597226011ad None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.706910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 412/828] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:31.708951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26a669-eb4b-496b-b1d1-54701f91bfd5 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.709597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26a669-eb4b-496b-b1d1-54701f91bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.710103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26a669-eb4b-496b-b1d1-54701f91bfd5 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.712263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 413/829] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:31.713466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26a669-eb4b-496b-b1d1-54701f91bfd5 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.773516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba9d03-59ad-4f55-ac27-ee7934fd992b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:31.810625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d26315-3b49-475b-9931-9d28edd248a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.811247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d26315-3b49-475b-9931-9d28edd248a8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.811526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d26315-3b49-475b-9931-9d28edd248a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.811799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d26315-3b49-475b-9931-9d28edd248a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.812582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d26315-3b49-475b-9931-9d28edd248a8 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=41d9317e24714de1a310198bb35eb00b) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:31.812888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d26315-3b49-475b-9931-9d28edd248a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.814349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d26315-3b49-475b-9931-9d28edd248a8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.821314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 417/830] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:31 2022] GET /identity/v3/role_assignments?user.id=41d9317e24714de1a310198bb35eb00b&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 610 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:31.822566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1099a4-ff50-4eb1-8d10-a3c756ca8b78 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.822848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1099a4-ff50-4eb1-8d10-a3c756ca8b78 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.823119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1099a4-ff50-4eb1-8d10-a3c756ca8b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.838732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26a669-eb4b-496b-b1d1-54701f91bfd5 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:31.845613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ec1099a4-ff50-4eb1-8d10-a3c756ca8b78 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:31.847173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ec1099a4-ff50-4eb1-8d10-a3c756ca8b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:31.881288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:31.882460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:31.882945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:31.883468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:31.891871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:31.894632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:31.908225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:31.909964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:31.911877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:31.911877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:31.911877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:31.911877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:31.911877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:31.911877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:31.911877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:31.911877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:31.914909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 414/831] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:31.915870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.930021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:31.954118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 418/832] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:31 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:31.977475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1099a4-ff50-4eb1-8d10-a3c756ca8b78 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:31.996229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:32.055698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd484-8330-41f3-9c32-138a875abe5a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:32.094943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:32.095574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.095857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.096131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.098800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:32.098800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:32.099463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:32.104843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1099a4-ff50-4eb1-8d10-a3c756ca8b78 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:32.115357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 415/833] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:32.130213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:32.145706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758f82-63ea-4d29-a7e2-57cf785065a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:32.146354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758f82-63ea-4d29-a7e2-57cf785065a6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.146630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758f82-63ea-4d29-a7e2-57cf785065a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.146898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758f82-63ea-4d29-a7e2-57cf785065a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.160770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1758f82-63ea-4d29-a7e2-57cf785065a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a1758f82-63ea-4d29-a7e2-57cf785065a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:32.167753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:32.168682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:32.169564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 419/834] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:32.175800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758f82-63ea-4d29-a7e2-57cf785065a6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:32.232146 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:32.261208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:32.274633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8662d75-1c49-42a4-abea-bc74a3d2f6d3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:32.290241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758f82-63ea-4d29-a7e2-57cf785065a6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:32.312124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c11b2-7eec-4c9c-9613-1ea54fde60cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:32.312984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c11b2-7eec-4c9c-9613-1ea54fde60cf None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.313384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c11b2-7eec-4c9c-9613-1ea54fde60cf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.313761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c11b2-7eec-4c9c-9613-1ea54fde60cf None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/212a75e45f2b449398acbfb80338afb0/roles/7329827bdc90439cb95f7d079698b690` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.329208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:32.330310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.330440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c11b2-7eec-4c9c-9613-1ea54fde60cf None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=212a75e45f2b449398acbfb80338afb0, role_id=7329827bdc90439cb95f7d079698b690)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:32.330906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.331896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.332778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:32.332882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c11b2-7eec-4c9c-9613-1ea54fde60cf None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:32.333350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:32.334895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:32.349211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 420/835] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:32.350370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:32.357761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 416/836] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:32 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/212a75e45f2b449398acbfb80338afb0/roles/7329827bdc90439cb95f7d079698b690 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:32.471703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268788b-8e86-4419-9c88-8019268cac32 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:32.512019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4968b97-e577-42d4-b446-da5931386d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:32.512587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4968b97-e577-42d4-b446-da5931386d46 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.512867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4968b97-e577-42d4-b446-da5931386d46 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.513145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4968b97-e577-42d4-b446-da5931386d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.516437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:32.526256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4968b97-e577-42d4-b446-da5931386d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:32.527726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4968b97-e577-42d4-b446-da5931386d46 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-e4968b97-e577-42d4-b446-da5931386d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:32.544247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:32.545176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:32.546098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:32.546098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:32.546098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:32.546098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:32.546098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 421/837] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:32.551668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c11b2-7eec-4c9c-9613-1ea54fde60cf None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:32.632947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ded0f-688a-4708-b9b4-af0ff18c2215 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.633279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ded0f-688a-4708-b9b4-af0ff18c2215 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.633530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ded0f-688a-4708-b9b4-af0ff18c2215 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.635039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 422/838] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:32.640391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e089921-3e7e-41ad-8257-3cc7e4cc6b33 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.640800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e089921-3e7e-41ad-8257-3cc7e4cc6b33 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.641078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e089921-3e7e-41ad-8257-3cc7e4cc6b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.660472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c11b2-7eec-4c9c-9613-1ea54fde60cf None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:32.660984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6e089921-3e7e-41ad-8257-3cc7e4cc6b33 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:32.662205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6e089921-3e7e-41ad-8257-3cc7e4cc6b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:32.688162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:32.708348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:32.708988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.709731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.710015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.711013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:32.711324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:32.713364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:32.729480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 417/839] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:32.732714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 423/840] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:32 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:32.736257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:32.755256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e089921-3e7e-41ad-8257-3cc7e4cc6b33 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:32.849008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c050e-76be-4001-bfeb-b047ec9b43c7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:32.887691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:32.888493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.888868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.889242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.889975 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e089921-3e7e-41ad-8257-3cc7e4cc6b33 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:32.899539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:32.901301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:32.912294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:32.913324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:32.914493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 22:08:32.915683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:32.921900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 418/841] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:32.924398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:32.939088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:32.939821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:32.940103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:32.940895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:32.941766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:32.943213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:32.957059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:32.958189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:32.959025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:32.963406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 424/842] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:32.964271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:32.975056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:33.041865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec37da-2a7b-42bd-a681-14a6dece3f02 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:33.074891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad6e1-7a91-4f4b-95c6-a568870da38d None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:33.078309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:33.079066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:33.079437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:33.079803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:33.080696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:33.081780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:33.083352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:33.107131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 419/843] 10.222.0.109 () {56 vars in 1140 bytes} [Thu Aug 4 22:08:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:33.113807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:33.114652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:33.114916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:33.115230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:33.115710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:33.116549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:33.117010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:33.118795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:33.129784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 425/844] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:32 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:33.134829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:33.186935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:33.224041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:33.230187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216eb4c-1761-4360-8318-642d386a8950 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:33.251117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cde52-349b-4917-91d8-0530f4c2120c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:33.266783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837ddf0-3ff4-4c7e-8c71-9ef5c5bac481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:33.267733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ddf0-3ff4-4c7e-8c71-9ef5c5bac481 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:33.268109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ddf0-3ff4-4c7e-8c71-9ef5c5bac481 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:33.268496 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ddf0-3ff4-4c7e-8c71-9ef5c5bac481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:33.289803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:33.290634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:33.291225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:33.291665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:33.292527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:33.293015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:33.294834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:33.307341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 426/845] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:33 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:33.343145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837ddf0-3ff4-4c7e-8c71-9ef5c5bac481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:33.344605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837ddf0-3ff4-4c7e-8c71-9ef5c5bac481 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:33.448191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:33.455093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 420/846] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:33 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:08:33.461140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:33.569043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6508f82-307b-4b2b-9349-b9be624cb200 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:33.580904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:33.610750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:33.611757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:33.612329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:33.613446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:33.624980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:33.626736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:33.647000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:33.647942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:33.648836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:33.648836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:33.648836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:33.648836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:33.648836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 427/847] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:33.653818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837ddf0-3ff4-4c7e-8c71-9ef5c5bac481 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:33.761676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837ddf0-3ff4-4c7e-8c71-9ef5c5bac481 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:33.796753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:33.797377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:33.797677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:33.797945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:33.798632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:33.798933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:33.800968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:33.813226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 421/848] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:33.819006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:33.930377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1044f8-34ea-4827-9a77-62cff0e5b40b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:33.966884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ae251-931d-4035-8902-09ba5cca9f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:33.967530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ae251-931d-4035-8902-09ba5cca9f18 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:33.967808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ae251-931d-4035-8902-09ba5cca9f18 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:33.968082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ae251-931d-4035-8902-09ba5cca9f18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:33.975780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ae251-931d-4035-8902-09ba5cca9f18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:33.977251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ae251-931d-4035-8902-09ba5cca9f18 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-8e3ae251-931d-4035-8902-09ba5cca9f18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:33.990215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:33.991663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:33.993175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 428/849] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:33.997123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:34.044467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f53c78-1200-4f87-bdf8-c0711a031483 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.044756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f53c78-1200-4f87-bdf8-c0711a031483 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.045045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f53c78-1200-4f87-bdf8-c0711a031483 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.046578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 429/850] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:34.056385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:34.056914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3319a7e-747b-434d-9a5a-95f158293ec3 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.058037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3319a7e-747b-434d-9a5a-95f158293ec3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.058465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3319a7e-747b-434d-9a5a-95f158293ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.079948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e3319a7e-747b-434d-9a5a-95f158293ec3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:34.082027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e3319a7e-747b-434d-9a5a-95f158293ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:34.108417 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:34.110863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c272-db4d-4914-a20a-277259b310a8 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:34.150775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:34.151498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.151841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.152176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.152991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:34.153359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:34.154894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:34.167867 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 422/851] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:34.169353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 430/852] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:34 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:34.174469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:34.195750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3319a7e-747b-434d-9a5a-95f158293ec3 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:34.280744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460b004-117a-439b-bf90-a9da70418e0c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:34.298005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:34.308080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3319a7e-747b-434d-9a5a-95f158293ec3 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:34.318431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:34.319379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.319861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.320325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.321327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=848151a99fb74f6b8f8210cfc49afeff) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:34.321860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:34.323474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:34.330087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 423/853] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:34 2022] GET /identity/v3/role_assignments?user.id=848151a99fb74f6b8f8210cfc49afeff&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 610 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:34.351826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:34.352517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.352851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.353132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.353843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:34.355290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:34.366472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:34.367485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:34.368786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 431/854] 10.222.0.109 () {56 vars in 1137 bytes} [Thu Aug 4 22:08:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:34.376474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:34.491268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e92650-aa27-4b5e-9549-1dd6e9812628 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:34.527010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105b0cf-e1ca-4727-a109-8e6e6c690232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:34.528020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105b0cf-e1ca-4727-a109-8e6e6c690232 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.528499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105b0cf-e1ca-4727-a109-8e6e6c690232 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.528996 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105b0cf-e1ca-4727-a109-8e6e6c690232 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.529952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105b0cf-e1ca-4727-a109-8e6e6c690232 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:34.530470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105b0cf-e1ca-4727-a109-8e6e6c690232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:34.532080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105b0cf-e1ca-4727-a109-8e6e6c690232 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:34.542697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 424/855] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:34 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:34.545918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:34.568823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:34.583173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235331-578a-4eba-b614-92ec759560fa None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.583493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235331-578a-4eba-b614-92ec759560fa None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.583730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235331-578a-4eba-b614-92ec759560fa None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.585331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 425/856] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:34.590338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b54e58-c10c-4711-8d1b-a69bd5f520e7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.590546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b54e58-c10c-4711-8d1b-a69bd5f520e7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.590821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b54e58-c10c-4711-8d1b-a69bd5f520e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.610361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-05b54e58-c10c-4711-8d1b-a69bd5f520e7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:34.611552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-05b54e58-c10c-4711-8d1b-a69bd5f520e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:34.663184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3edb-be5c-4fe1-9b84-1f0bf907e6ec None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:34.685442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:34.702424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:34.703054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.703336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.703612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.704296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:34.704597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:34.705482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 426/857] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:34 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:34.706696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:34.711821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 432/858] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:34 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:34.725814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b54e58-c10c-4711-8d1b-a69bd5f520e7 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:34.839602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b54e58-c10c-4711-8d1b-a69bd5f520e7 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:34.877310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada4ffb-a84c-4430-932c-876c6203e660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:34.878029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada4ffb-a84c-4430-932c-876c6203e660 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:34.878300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada4ffb-a84c-4430-932c-876c6203e660 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:34.878577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada4ffb-a84c-4430-932c-876c6203e660 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:34.889210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada4ffb-a84c-4430-932c-876c6203e660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-eada4ffb-a84c-4430-932c-876c6203e660 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:34.895226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:34.896177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:34.897395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 427/859] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:08:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:34.902437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:35.000307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:35.017954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c24f30-76dd-48c2-8557-813e4b8ddc78 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:35.055101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:35.055733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.056022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.056292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:35.057002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:35.057292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:35.058719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:35.068543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 433/860] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:08:34 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:35.074498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada4ffb-a84c-4430-932c-876c6203e660 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:35.111773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:35.185779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada4ffb-a84c-4430-932c-876c6203e660 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:35.221969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:35.222990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.223505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.223970 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:35.234701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:35.236435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:35.253537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:35.254434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:35.255319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:35.255319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:35.255319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:35.255319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:35.261504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 428/861] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:35.263244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:35.368388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e65e2-a56e-4b5c-b8e8-543437f39935 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:35.406717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2175a5-5ff2-4bba-804b-75424865b707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:35.407731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2175a5-5ff2-4bba-804b-75424865b707 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.408211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2175a5-5ff2-4bba-804b-75424865b707 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.408707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2175a5-5ff2-4bba-804b-75424865b707 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:35.409658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2175a5-5ff2-4bba-804b-75424865b707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:35.410240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2175a5-5ff2-4bba-804b-75424865b707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:35.411871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2175a5-5ff2-4bba-804b-75424865b707 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:35.424469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 434/862] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:35.431034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:35.455068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a84974-2e0f-440c-8b03-5250b28e4658 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.455371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a84974-2e0f-440c-8b03-5250b28e4658 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.455641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a84974-2e0f-440c-8b03-5250b28e4658 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:35.457768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 435/863] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:35.463851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d0cc8-0080-48a9-aea8-946c710aba87 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.464026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d0cc8-0080-48a9-aea8-946c710aba87 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.464242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d0cc8-0080-48a9-aea8-946c710aba87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:35.470629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:35.482781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ac0d0cc8-0080-48a9-aea8-946c710aba87 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:35.483980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ac0d0cc8-0080-48a9-aea8-946c710aba87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:35.529792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:35.546768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b53f11-6501-4472-919c-29af72aa69f9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:35.564040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 436/864] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:35 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:35.587867 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:35.589115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d0cc8-0080-48a9-aea8-946c710aba87 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:35.590001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.590522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.590995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:35.599369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:35.601101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:35.613197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:35.614189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:35.615133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:35.617721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 429/865] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:08:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:35.622912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:35.713159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d0cc8-0080-48a9-aea8-946c710aba87 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:35.742525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abbcab-d50c-47bb-b871-47abd0dc5f5f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:35.748003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:35.755110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c2bd-cc87-4a7f-bc60-fb48d6ff9112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:35.756186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c2bd-cc87-4a7f-bc60-fb48d6ff9112 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.756738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c2bd-cc87-4a7f-bc60-fb48d6ff9112 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.757244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c2bd-cc87-4a7f-bc60-fb48d6ff9112 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:35.758240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900c2bd-cc87-4a7f-bc60-fb48d6ff9112 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:35.760118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900c2bd-cc87-4a7f-bc60-fb48d6ff9112 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:35.778605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 437/866] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:08:35 2022] POST /identity/v3/services => generated 252 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:35.800940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:35.801683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.802018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.802336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:35.803191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:35.803557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:35.805158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:35.822319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:35.826937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 430/867] 10.222.0.109 () {56 vars in 1154 bytes} [Thu Aug 4 22:08:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:35.832930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c2bd-cc87-4a7f-bc60-fb48d6ff9112 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:35.958210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c2bd-cc87-4a7f-bc60-fb48d6ff9112 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:35.993476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68235da4-2129-436d-a50c-3524e7bf9ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:35.994234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68235da4-2129-436d-a50c-3524e7bf9ca3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:35.994515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68235da4-2129-436d-a50c-3524e7bf9ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:35.994790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68235da4-2129-436d-a50c-3524e7bf9ca3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:36.057496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:36.075305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68235da4-2129-436d-a50c-3524e7bf9ca3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:36.077127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68235da4-2129-436d-a50c-3524e7bf9ca3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:36.187117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 438/868] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:35 2022] GET /identity/v3/auth/tokens => generated 1264 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:08:36.193783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:36.301066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75d963-c072-4e98-9971-46e5f7b5c9b0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:36.337324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc77b2d-bd75-4491-aebf-b24fb091520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:36.338360 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77b2d-bd75-4491-aebf-b24fb091520e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:36.338837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77b2d-bd75-4491-aebf-b24fb091520e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:36.339305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77b2d-bd75-4491-aebf-b24fb091520e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:36.350776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc77b2d-bd75-4491-aebf-b24fb091520e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:36.352492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc77b2d-bd75-4491-aebf-b24fb091520e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-2cc77b2d-bd75-4491-aebf-b24fb091520e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:36.367934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:36.368854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:36.369774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:36.369774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:36.369774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:36.369774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:36.371924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 431/869] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:36.377590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68235da4-2129-436d-a50c-3524e7bf9ca3 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:36.472573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:36.481544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68235da4-2129-436d-a50c-3524e7bf9ca3 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:36.498160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10e2e-d925-47f3-9a8f-ca99f86535ab None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:36.498667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10e2e-d925-47f3-9a8f-ca99f86535ab None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:36.498744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10e2e-d925-47f3-9a8f-ca99f86535ab None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:36.500294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 432/870] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:36.505964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039a71-c393-4ab7-8a8f-5a64c78fe738 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:36.506248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039a71-c393-4ab7-8a8f-5a64c78fe738 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:36.506525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039a71-c393-4ab7-8a8f-5a64c78fe738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:36.519373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:36.519974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:36.520686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:36.520686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:36.521212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:36.521542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:36.522941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:36.526477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-aa039a71-c393-4ab7-8a8f-5a64c78fe738 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:36.527872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-aa039a71-c393-4ab7-8a8f-5a64c78fe738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:36.536698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 439/871] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:36.547159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:36.589356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:36.621487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 433/872] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:36 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:36.645145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa039a71-c393-4ab7-8a8f-5a64c78fe738 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:36.682111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b742a0-72dd-4ef3-b22f-1a731e259081 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:36.724023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:36.724654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:36.724888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:36.725123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:36.732507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:36.734190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:36.747343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:36.748270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:36.749151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:36.749151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:36.749151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:36.749151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:36.749151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:36.749151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:36.749151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:36.749151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:36.753987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 440/873] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:36.754913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:36.764197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa039a71-c393-4ab7-8a8f-5a64c78fe738 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:36.772023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:36.802458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:36.803444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:36.803945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:36.804409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:36.805225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:36.807091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:36.817711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:36.818855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:36.819685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:36.822618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 434/874] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:36.828515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:36.883061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecde8c-fb1d-44e4-adaf-13ec8b1ebcd3 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:36.891800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:36.929161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:36.930047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:36.930516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:36.930967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:36.932273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:36.932789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:36.935111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:36.957435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 441/875] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:36.958846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:36.965243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e45383-1c52-4f5c-b665-5f996965e65a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:37.007942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c5d14-75b3-4dc7-ac93-04c4d095b12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:37.008897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c5d14-75b3-4dc7-ac93-04c4d095b12d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:37.009179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c5d14-75b3-4dc7-ac93-04c4d095b12d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:37.009781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c5d14-75b3-4dc7-ac93-04c4d095b12d None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:37.010746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c5d14-75b3-4dc7-ac93-04c4d095b12d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:37.011054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c5d14-75b3-4dc7-ac93-04c4d095b12d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:37.012923 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c5d14-75b3-4dc7-ac93-04c4d095b12d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:37.023725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 435/876] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:36 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:37.023725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152d53-17ae-4b7b-9e22-4c862a3f4e3e None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:37.023725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152d53-17ae-4b7b-9e22-4c862a3f4e3e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:37.023725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152d53-17ae-4b7b-9e22-4c862a3f4e3e None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:37.025755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 436/877] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:37 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:37.026279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152d53-17ae-4b7b-9e22-4c862a3f4e3e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:37.041316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:37.075574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b96316-ee65-4853-9ed9-de4d4680da04 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:37.119800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a50e89-f6c5-48e8-bb97-08539d4f8080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:37.120423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a50e89-f6c5-48e8-bb97-08539d4f8080 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:37.120701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a50e89-f6c5-48e8-bb97-08539d4f8080 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:37.120972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a50e89-f6c5-48e8-bb97-08539d4f8080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:37.121793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a50e89-f6c5-48e8-bb97-08539d4f8080 None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=212a75e45f2b449398acbfb80338afb0) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:37.122096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a50e89-f6c5-48e8-bb97-08539d4f8080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:37.123541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a50e89-f6c5-48e8-bb97-08539d4f8080 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:37.131364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 442/878] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:36 2022] GET /identity/v3/role_assignments?user.id=212a75e45f2b449398acbfb80338afb0&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 610 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:37.132520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed2b9b-de1c-4ef3-aa3a-5c575cb9cd9e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:37.132807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed2b9b-de1c-4ef3-aa3a-5c575cb9cd9e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:37.133134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed2b9b-de1c-4ef3-aa3a-5c575cb9cd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:37.149991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152d53-17ae-4b7b-9e22-4c862a3f4e3e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:37.152948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a9ed2b9b-de1c-4ef3-aa3a-5c575cb9cd9e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:37.154355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a9ed2b9b-de1c-4ef3-aa3a-5c575cb9cd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:37.166728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:37.205455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:37.206593 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:37.207836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:37.208332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:37.209740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:37.210537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:37.212440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:37.220521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 437/879] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:37 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:37.249646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 443/880] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:37 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:37.267385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:37.392064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43fd21-2057-40f6-8c30-66420329cd39 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:37.428016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40bb07-5ccd-46fa-b585-99d7fc7f8d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:37.428642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40bb07-5ccd-46fa-b585-99d7fc7f8d00 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:37.428917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40bb07-5ccd-46fa-b585-99d7fc7f8d00 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:37.429188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40bb07-5ccd-46fa-b585-99d7fc7f8d00 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:37.429907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40bb07-5ccd-46fa-b585-99d7fc7f8d00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:37.431262 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40bb07-5ccd-46fa-b585-99d7fc7f8d00 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:37.450848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 438/881] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:08:37 2022] POST /identity/v3/services => generated 270 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:37.894057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935f83-15e4-462c-be86-5ead05e6a077 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:37.894659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935f83-15e4-462c-be86-5ead05e6a077 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:37.894659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935f83-15e4-462c-be86-5ead05e6a077 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:37.896400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 444/882] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:37.901626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909851f-c82e-437d-a94e-c8d5b5516840 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:37.902051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909851f-c82e-437d-a94e-c8d5b5516840 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:37.902229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909851f-c82e-437d-a94e-c8d5b5516840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:37.922069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6909851f-c82e-437d-a94e-c8d5b5516840 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:37.923252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6909851f-c82e-437d-a94e-c8d5b5516840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:37.951618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:38.002112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 439/883] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:37 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:38.024557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a935f83-15e4-462c-be86-5ead05e6a077 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:38.092651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:38.130676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a935f83-15e4-462c-be86-5ead05e6a077 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:38.166586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:38.167147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:38.167425 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:38.167697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:38.168327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:38.169727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:38.179931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:38.180928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:38.181802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 445/884] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:38.186930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909851f-c82e-437d-a94e-c8d5b5516840 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:38.295013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909851f-c82e-437d-a94e-c8d5b5516840 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:38.330289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:38.331827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:38.332290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:38.332739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:38.333621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:38.334124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:38.335665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:38.340917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 440/885] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:38 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:38.346422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:38.453604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c4a74-3ce6-4f3b-a748-026071bfb938 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:38.481399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:38.488136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:38.489106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:38.489767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:38.490037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:38.490707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:38.491006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:38.492377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:38.497520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 446/886] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:38 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:38.503607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:38.529169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:38.608346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e4b3-7e4b-4215-b842-571745043e79 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:38.647796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1e005-500c-418e-9ec9-8eab87fac050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:38.648637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1e005-500c-418e-9ec9-8eab87fac050 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:38.649064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1e005-500c-418e-9ec9-8eab87fac050 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:38.649506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1e005-500c-418e-9ec9-8eab87fac050 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:38.650424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1e005-500c-418e-9ec9-8eab87fac050 None admin] RBAC: Adding query filter params (interface=public, service_id=ddc783caeb684bddb22817f35511a3f1, region_id=RegionOne) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:38.650890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1e005-500c-418e-9ec9-8eab87fac050 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:38.652394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1e005-500c-418e-9ec9-8eab87fac050 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:38.658074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 441/887] 10.222.0.109 () {56 vars in 1256 bytes} [Thu Aug 4 22:08:38 2022] GET /identity/v3/endpoints?service_id=ddc783caeb684bddb22817f35511a3f1&interface=public®ion_id=RegionOne => generated 195 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:38.663497 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:38.770233 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ad14b-9af1-4e81-ae1b-fb792ff9da08 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:38.806746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63cce6-a9fa-4ec4-b95b-8ad820b9c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:38.807737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63cce6-a9fa-4ec4-b95b-8ad820b9c4c1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:38.808224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63cce6-a9fa-4ec4-b95b-8ad820b9c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:38.808700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63cce6-a9fa-4ec4-b95b-8ad820b9c4c1 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:38.809732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63cce6-a9fa-4ec4-b95b-8ad820b9c4c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:38.811329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63cce6-a9fa-4ec4-b95b-8ad820b9c4c1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:38.818413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 447/888] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:08:38 2022] GET /identity/v3/services => generated 1274 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:08:39.377091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54372131-911e-430b-9c36-8c0c279d21b7 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.378269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54372131-911e-430b-9c36-8c0c279d21b7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.378269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54372131-911e-430b-9c36-8c0c279d21b7 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.379909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 442/889] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:39.381254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e4740-63c0-46af-b011-8a941357e897 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.381658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e4740-63c0-46af-b011-8a941357e897 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.381847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e4740-63c0-46af-b011-8a941357e897 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.383276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 448/890] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:39.388177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c0567-0d18-4035-8dbe-30f830cbd182 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.388459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c0567-0d18-4035-8dbe-30f830cbd182 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.388733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c0567-0d18-4035-8dbe-30f830cbd182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.389994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f72d3-0d36-4bde-ae01-9ad5c683b196 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.390509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f72d3-0d36-4bde-ae01-9ad5c683b196 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.390982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f72d3-0d36-4bde-ae01-9ad5c683b196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.409190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-449c0567-0d18-4035-8dbe-30f830cbd182 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:39.413141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-449c0567-0d18-4035-8dbe-30f830cbd182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:39.414300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7c8f72d3-0d36-4bde-ae01-9ad5c683b196 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:39.416931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7c8f72d3-0d36-4bde-ae01-9ad5c683b196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:39.476172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:39.492911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 449/891] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:39 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:39.508008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 443/892] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:39 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:39.516459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f72d3-0d36-4bde-ae01-9ad5c683b196 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:39.546676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c0567-0d18-4035-8dbe-30f830cbd182 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:39.570756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:39.634202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f72d3-0d36-4bde-ae01-9ad5c683b196 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:39.670650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70d895-0c51-45c0-845e-f1b0e3442aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:39.671593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70d895-0c51-45c0-845e-f1b0e3442aec None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.672084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70d895-0c51-45c0-845e-f1b0e3442aec None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.672576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70d895-0c51-45c0-845e-f1b0e3442aec None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.673509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70d895-0c51-45c0-845e-f1b0e3442aec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:39.675890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70d895-0c51-45c0-845e-f1b0e3442aec None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:39.683909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c0567-0d18-4035-8dbe-30f830cbd182 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:39.691584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 450/893] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:08:39 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:39.692887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050621-87b4-4004-abe0-869052ef20c5 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.693178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050621-87b4-4004-abe0-869052ef20c5 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.693479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050621-87b4-4004-abe0-869052ef20c5 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.694884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 451/894] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:39.700353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9bc9c-3ffd-4c80-ac53-bc7254c7640e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.700661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9bc9c-3ffd-4c80-ac53-bc7254c7640e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.700956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9bc9c-3ffd-4c80-ac53-bc7254c7640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.719933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-1fa9bc9c-3ffd-4c80-ac53-bc7254c7640e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:39.721706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1fa9bc9c-3ffd-4c80-ac53-bc7254c7640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:39.730704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac378d-68b6-429a-9c0f-40ecb5f5f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:39.731820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac378d-68b6-429a-9c0f-40ecb5f5f6ab None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.732308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac378d-68b6-429a-9c0f-40ecb5f5f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.732829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac378d-68b6-429a-9c0f-40ecb5f5f6ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.746297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac378d-68b6-429a-9c0f-40ecb5f5f6ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-68ac378d-68b6-429a-9c0f-40ecb5f5f6ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:39.752122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:39.754060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:08:39.754904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:39.762748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 444/895] 10.222.0.109 () {56 vars in 1125 bytes} [Thu Aug 4 22:08:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:39.764137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac378d-68b6-429a-9c0f-40ecb5f5f6ab None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:39.772915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:39.803761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 452/896] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:39 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:39.827262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9bc9c-3ffd-4c80-ac53-bc7254c7640e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:39.881308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:39.919432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac378d-68b6-429a-9c0f-40ecb5f5f6ab None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:39.950862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9bc9c-3ffd-4c80-ac53-bc7254c7640e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:39.961858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:39.962875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.963395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.963894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:39.964816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:39.965429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:39.967059 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:39.978629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 445/897] 10.222.0.109 () {56 vars in 1120 bytes} [Thu Aug 4 22:08:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:39.993434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:39.998474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:39.999128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:39.999405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:39.999677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.000307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.001735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:40.011950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:40.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:40.013831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 453/898] 10.222.0.109 () {56 vars in 1137 bytes} [Thu Aug 4 22:08:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:40.019616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:40.065636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:40.091119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8fe91-8f35-4aa2-876e-785b1e4a9082 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.136505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.137103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.137381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.137690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.140792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:40.147500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5312-ba35-480e-9ffe-8d9c1f439c6b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.151194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.152741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:40.168424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:40.169345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:40.170315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:40.170315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:40.170315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:40.170315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:40.174549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 446/899] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:40.175917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:40.195066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.195686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.195961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.196228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.196897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:40.197194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.198634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.213185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 454/900] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:40 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:40.214244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:40.317169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f198d1-6493-4725-8ab9-d0bfcddab932 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.323526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0ada5-0915-48c2-bd06-c7028b42483e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.333941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:40.356782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.357418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.357720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.358267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.358773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:40.359074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.360466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.372485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.373441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.373949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.374423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.375302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:40.375803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.377418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.377821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 447/901] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:40.379938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:40.390365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 455/902] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:40 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:40.393476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:40.445504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:40.499586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dcf11-280b-40a1-808f-42a28d033f88 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.514847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39640df0-97b4-44c3-8c25-5b4005173f37 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.537111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.538085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.538538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.538982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.539919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] RBAC: Adding query filter params (interface=public, service_id=35a667fe04974262893439707352f587, region_id=RegionOne) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:40.540397 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.542193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.548016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 456/903] 10.222.0.109 () {56 vars in 1256 bytes} [Thu Aug 4 22:08:40 2022] GET /identity/v3/endpoints?service_id=35a667fe04974262893439707352f587&interface=public®ion_id=RegionOne => generated 195 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:40.553291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:40.569270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.570581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.571066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.571767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.583281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.585610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.592478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:40.603157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:40.604102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:40.605015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:40.611662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 448/904] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:08:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:40.612938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:40.687778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f092c-b4da-4886-aad8-69918d697ff1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.728731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fc5bb-99ef-4e00-b8e4-f1f963d2c056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.730135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc5bb-99ef-4e00-b8e4-f1f963d2c056 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.730665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc5bb-99ef-4e00-b8e4-f1f963d2c056 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.731107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc5bb-99ef-4e00-b8e4-f1f963d2c056 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.732162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257fc5bb-99ef-4e00-b8e4-f1f963d2c056 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.734349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257fc5bb-99ef-4e00-b8e4-f1f963d2c056 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.735605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a6368-b896-4c4b-b0a8-08a0d25821ca None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.750088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 457/905] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:08:40 2022] GET /identity/v3/services => generated 1504 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:08:40.753086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:40.782310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b52b61-cfaa-462a-91f2-3da899a65a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.782942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52b61-cfaa-462a-91f2-3da899a65a95 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.783216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52b61-cfaa-462a-91f2-3da899a65a95 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.783483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52b61-cfaa-462a-91f2-3da899a65a95 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.784949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b52b61-cfaa-462a-91f2-3da899a65a95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:40.785334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b52b61-cfaa-462a-91f2-3da899a65a95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:40.786809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b52b61-cfaa-462a-91f2-3da899a65a95 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:40.807610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 449/906] 10.222.0.109 () {56 vars in 1154 bytes} [Thu Aug 4 22:08:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:40.813670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fc5bb-99ef-4e00-b8e4-f1f963d2c056 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:40.900174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced1f38-1b20-4ef7-869e-e053e3bcbf81 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.900552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced1f38-1b20-4ef7-869e-e053e3bcbf81 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.900735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced1f38-1b20-4ef7-869e-e053e3bcbf81 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.903752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 450/907] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:40.909242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e845e1-35b7-4b2a-a35a-3e6b2b23df0d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.909574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e845e1-35b7-4b2a-a35a-3e6b2b23df0d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.909820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e845e1-35b7-4b2a-a35a-3e6b2b23df0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:40.932338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-80e845e1-35b7-4b2a-a35a-3e6b2b23df0d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:40.932722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fc5bb-99ef-4e00-b8e4-f1f963d2c056 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:40.934074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-80e845e1-35b7-4b2a-a35a-3e6b2b23df0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:40.948470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:40.980981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af15af9-8445-475f-b2f0-4486ced2d84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:40.981598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15af9-8445-475f-b2f0-4486ced2d84e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:40.981884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15af9-8445-475f-b2f0-4486ced2d84e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:40.982141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15af9-8445-475f-b2f0-4486ced2d84e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.031676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 451/908] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:40 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:41.061565 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e845e1-35b7-4b2a-a35a-3e6b2b23df0d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.064976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af15af9-8445-475f-b2f0-4486ced2d84e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.066498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af15af9-8445-475f-b2f0-4486ced2d84e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.169352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:41.177198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 458/909] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:40 2022] GET /identity/v3/auth/tokens => generated 1476 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:08:41.182547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:41.185575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af15af9-8445-475f-b2f0-4486ced2d84e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.189710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e845e1-35b7-4b2a-a35a-3e6b2b23df0d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:41.234691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:41.235948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.236754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.237756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.238988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.240589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:41.254435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:41.255440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 22:08:41.256277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:41.259511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 452/910] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:41.266113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.301921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af15af9-8445-475f-b2f0-4486ced2d84e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:41.347300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:41.347300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.347300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.347300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.357067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.359277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:41.376547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:41.377545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:41.378690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:41.378690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:41.378690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:41.378690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:41.382651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 459/911] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:41.383672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.396490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436b415-f529-4efe-83a3-39ee263bc398 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:41.440834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:41.441671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.442182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:41.442642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.442642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.443623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:41.443929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.445732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.458592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 453/912] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:41 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:41.460650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.488166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:41.514753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ba74-c083-4ba2-bb95-61aacc071640 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:41.563817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:41.564692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.565472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.565889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.567393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:41.567840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.570533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.573081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3f8da-5c41-4c39-99df-826482c0c0ec None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:41.591198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 460/913] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:41.592673 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.613057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:41.614061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.614569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.615039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.615910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:41.616408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.618012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.628922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 454/914] 10.222.0.109 () {56 vars in 1119 bytes} [Thu Aug 4 22:08:41 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:41.630112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.719869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110a53a-8da7-450a-a1d5-58b103e1dec8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:41.729334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:41.733128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:41.747484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd21092-d41b-4d3c-b363-d15e71bd454a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:41.760788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:41.761746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.762435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.763101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.770914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.772619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.786149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62379d8d-882d-4f09-9907-4a5521468f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:41.786764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62379d8d-882d-4f09-9907-4a5521468f48 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:41.787009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:41.787921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:41.788809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:41.788809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:41.788809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:41.788809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:41.788809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:41.788809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:41.788809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:41.788809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:41.789225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62379d8d-882d-4f09-9907-4a5521468f48 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.789225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62379d8d-882d-4f09-9907-4a5521468f48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.789225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62379d8d-882d-4f09-9907-4a5521468f48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.789432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62379d8d-882d-4f09-9907-4a5521468f48 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.796061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 461/915] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:41.797932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.825035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 455/916] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:08:41 2022] POST /identity/v3/endpoints => generated 348 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:41.827248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5b6f-c208-4c9f-85de-89f13db2f6fd None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.827541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5b6f-c208-4c9f-85de-89f13db2f6fd None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.827815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5b6f-c208-4c9f-85de-89f13db2f6fd None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.829344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 456/917] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:41.835447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc4b63-917f-4ea6-9cfc-5e1aa11da49d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.835750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc4b63-917f-4ea6-9cfc-5e1aa11da49d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.835993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc4b63-917f-4ea6-9cfc-5e1aa11da49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.859623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-06cc4b63-917f-4ea6-9cfc-5e1aa11da49d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:41.861662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-06cc4b63-917f-4ea6-9cfc-5e1aa11da49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:41.926499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca831fd4-d2aa-43e4-bf47-169b2f64b470 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:41.969445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 457/918] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:41 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:41.980553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:41.981541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:41.981831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:41.982112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:41.982894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:41.983205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:41.984677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:41.993624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc4b63-917f-4ea6-9cfc-5e1aa11da49d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:41.999626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 462/919] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:42.006465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:42.042408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:42.093354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:42.138033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2eba52-1540-446b-8dc4-32c9a8dd7ab7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:42.143241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc4b63-917f-4ea6-9cfc-5e1aa11da49d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:42.185107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:42.185832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:42.186117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:42.186247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:42.186410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:42.186985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:42.187244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] RBAC: Adding query filter params (role.id=a449c88b1754437d942b32f2ec39cb9b, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=212a75e45f2b449398acbfb80338afb0) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:42.187348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:42.187550 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:42.187707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:42.188414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:42.188995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:42.190518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:42.196744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 463/920] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:42 2022] GET /identity/v3/role_assignments?user.id=212a75e45f2b449398acbfb80338afb0&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=a449c88b1754437d942b32f2ec39cb9b => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:42.202830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:42.203930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:42.205114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 458/921] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:42.211964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:42.318925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:42.329234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf1ea8-1e0f-4459-948a-605ea1a772a8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:42.366016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:42.366652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:42.366930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:42.367200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:42.367931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:42.368230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:42.369626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:42.375017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 464/922] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:42 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:42.380665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:42.485894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9dfb2-15cb-4869-8413-0d3b122143c3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:42.525724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:42.526765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:42.527256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:42.527767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:42.528647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:42.529150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:42.530793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:42.532419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:42.537982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 459/923] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:42 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:42.544237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:42.652328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354fcd8-08bc-48bf-961b-2ba677a7d977 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:42.689124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d766d-1792-493a-8c52-d978892d107b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:42.689804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d766d-1792-493a-8c52-d978892d107b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:42.690091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d766d-1792-493a-8c52-d978892d107b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:42.690364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d766d-1792-493a-8c52-d978892d107b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:42.691183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d766d-1792-493a-8c52-d978892d107b None admin] RBAC: Adding query filter params (interface=public, service_id=a5eefdf368ec4963ab4b8fcd8dc5994b, region_id=RegionOne) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:42.691490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d766d-1792-493a-8c52-d978892d107b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:42.692927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d766d-1792-493a-8c52-d978892d107b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:42.698660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 465/924] 10.222.0.109 () {56 vars in 1256 bytes} [Thu Aug 4 22:08:42 2022] GET /identity/v3/endpoints?service_id=a5eefdf368ec4963ab4b8fcd8dc5994b&interface=public®ion_id=RegionOne => generated 195 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:42.706817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:42.817636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af69032-a299-471f-8a6c-51afc7d76cdf None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:42.852437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06ea1d-e401-44fc-9f89-92c538c2d177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:42.853068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06ea1d-e401-44fc-9f89-92c538c2d177 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:42.853361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06ea1d-e401-44fc-9f89-92c538c2d177 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:42.853647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06ea1d-e401-44fc-9f89-92c538c2d177 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:42.854307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06ea1d-e401-44fc-9f89-92c538c2d177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:42.855664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06ea1d-e401-44fc-9f89-92c538c2d177 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:42.862562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 460/925] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:08:42 2022] GET /identity/v3/services => generated 1504 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:08:42.998173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f32d1-d7b1-40d4-b62f-ad9e225b7c04 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:42.998587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f32d1-d7b1-40d4-b62f-ad9e225b7c04 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:42.999020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f32d1-d7b1-40d4-b62f-ad9e225b7c04 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:43.000723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 466/926] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:42 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:43.008143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e192ec-c4d7-474d-8921-8158ae126eff None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:43.008143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e192ec-c4d7-474d-8921-8158ae126eff None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:43.008143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e192ec-c4d7-474d-8921-8158ae126eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:43.027214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-b6e192ec-c4d7-474d-8921-8158ae126eff None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:43.028408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b6e192ec-c4d7-474d-8921-8158ae126eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:43.093115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:43.112501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 461/927] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:43 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:43.127851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f32d1-d7b1-40d4-b62f-ad9e225b7c04 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:43.149919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:43.243252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f32d1-d7b1-40d4-b62f-ad9e225b7c04 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:43.279686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:43.280249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:43.280530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:43.280801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:43.281437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:43.282817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:43.292785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:43.293763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:43.294607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 467/928] 10.222.0.109 () {56 vars in 1137 bytes} [Thu Aug 4 22:08:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:43.300648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e192ec-c4d7-474d-8921-8158ae126eff None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:43.406423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e192ec-c4d7-474d-8921-8158ae126eff None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:43.445788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:43.446419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:43.446697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:43.446971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:43.447655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:43.447958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:43.449333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:43.454660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 462/929] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:43 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:43.460768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:43.548291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:43.569543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0de390-7c12-4fa9-953e-0916413feabb None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:43.606077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568dedca-2fd2-46cc-b36d-4ac226f5ab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:43.606701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dedca-2fd2-46cc-b36d-4ac226f5ab90 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:43.606981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dedca-2fd2-46cc-b36d-4ac226f5ab90 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:43.607250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dedca-2fd2-46cc-b36d-4ac226f5ab90 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:43.607926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dedca-2fd2-46cc-b36d-4ac226f5ab90 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:43.608227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dedca-2fd2-46cc-b36d-4ac226f5ab90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:43.609618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dedca-2fd2-46cc-b36d-4ac226f5ab90 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:43.615752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 468/930] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:43 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:43.621308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:43.684290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:43.731746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d5ce4-da78-4f0d-ac47-7a9299983efe None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:43.769495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a2fd4-ba9f-4612-883b-16fc80b55123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:43.770120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a2fd4-ba9f-4612-883b-16fc80b55123 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:43.770400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a2fd4-ba9f-4612-883b-16fc80b55123 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:43.770672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a2fd4-ba9f-4612-883b-16fc80b55123 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:43.771394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a2fd4-ba9f-4612-883b-16fc80b55123 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:43.772763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a2fd4-ba9f-4612-883b-16fc80b55123 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:43.794606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 463/931] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:08:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:44.070944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df68726-5cb6-49ff-9d01-265d8c58e6f7 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.071332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df68726-5cb6-49ff-9d01-265d8c58e6f7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.071507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df68726-5cb6-49ff-9d01-265d8c58e6f7 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.073104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 469/932] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:44.078199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf6645-fe11-4e04-b699-245092e611f3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.078489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf6645-fe11-4e04-b699-245092e611f3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.078766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf6645-fe11-4e04-b699-245092e611f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.097309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9bdf6645-fe11-4e04-b699-245092e611f3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:44.098551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9bdf6645-fe11-4e04-b699-245092e611f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:44.177711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 464/933] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:44 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:44.200876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df68726-5cb6-49ff-9d01-265d8c58e6f7 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:44.308530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df68726-5cb6-49ff-9d01-265d8c58e6f7 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:44.344941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:44.345598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.345906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.346181 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.346861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:44.348310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:44.350928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING keystone.server.flask.common [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 22:08:44.370496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.sql.core [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 22:08:44.370496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 22:08:44.370496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [parameters: {'id': 'e1fdd3c22d4e4df9b07a80a14356b4f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 22:08:44.370496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91701) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 22:08:44.378096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d430e3-8820-48f4-9b5e-8dac12c3ec63 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.378827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d430e3-8820-48f4-9b5e-8dac12c3ec63 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.379324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d430e3-8820-48f4-9b5e-8dac12c3ec63 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.381267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 465/934] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:44 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:44.386446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c53d58-51fa-4bc3-a250-c169d16637fd None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.386956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c53d58-51fa-4bc3-a250-c169d16637fd None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.387446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c53d58-51fa-4bc3-a250-c169d16637fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1fdd3c22d4e4df9b07a80a14356b4f0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1fdd3c22d4e4df9b07a80a14356b4f0. Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application conn.query(q) Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 22:08:44.388950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result.read() Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 22:08:44.389761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application next(self.gen) Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application yield resource Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application next(self.gen) Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise self.value Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application session.commit() Aug 04 22:08:44.390689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.session.flush() Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application compat.raise_( Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 22:08:44.391570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 22:08:44.392884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application conn.query(q) Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result.read() Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 22:08:44.395529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [parameters: {'id': 'e1fdd3c22d4e4df9b07a80a14356b4f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:44.397822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:44.399646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1fdd3c22d4e4df9b07a80a14356b4f0. Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 470/935] 10.222.0.109 () {60 vars in 1145 bytes} [Thu Aug 4 22:08:44 2022] POST /identity/v3/projects => generated 262 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:44.401747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:44.425529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-73c53d58-51fa-4bc3-a250-c169d16637fd None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:44.427598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-73c53d58-51fa-4bc3-a250-c169d16637fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:44.466484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:44.524348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea5111-7745-4e0e-8b06-685b111c89f3 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:44.531168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 466/936] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:44 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:44.554474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c53d58-51fa-4bc3-a250-c169d16637fd None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:44.562556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:44.563903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.564406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.565475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.573606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:44.575683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:44.590328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:44.591239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:44.592127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:44.592127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:44.592127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:44.592127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:44.592127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:44.592127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:44.592127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:44.592127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:44.594575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 471/937] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:44.600598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:44.661293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c53d58-51fa-4bc3-a250-c169d16637fd None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:44.668942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:44.698618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22670bb-a2b6-4014-9027-8726589bb812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:44.699265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22670bb-a2b6-4014-9027-8726589bb812 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.699542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22670bb-a2b6-4014-9027-8726589bb812 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.699810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22670bb-a2b6-4014-9027-8726589bb812 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.711149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22670bb-a2b6-4014-9027-8726589bb812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:44.715302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e55f5-1411-4157-9789-1b8a74e1cf4b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-a22670bb-a2b6-4014-9027-8726589bb812 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:44.716139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:44.717045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:08:44.717925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:44.722027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 467/938] 10.222.0.109 () {56 vars in 1125 bytes} [Thu Aug 4 22:08:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:44.724039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22670bb-a2b6-4014-9027-8726589bb812 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:44.753486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:44.758161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:44.759021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.759553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.760029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.760957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:44.761550 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:44.763192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:44.793766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 472/939] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:44 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:44.836509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22670bb-a2b6-4014-9027-8726589bb812 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:44.874738 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94722e86-5a18-4696-a901-0d4bf458686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:44.875269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94722e86-5a18-4696-a901-0d4bf458686e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:44.875542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94722e86-5a18-4696-a901-0d4bf458686e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:44.875809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94722e86-5a18-4696-a901-0d4bf458686e None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:44.876532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94722e86-5a18-4696-a901-0d4bf458686e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:44.876831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94722e86-5a18-4696-a901-0d4bf458686e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:44.878793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94722e86-5a18-4696-a901-0d4bf458686e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:44.888839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 468/940] 10.222.0.109 () {56 vars in 1120 bytes} [Thu Aug 4 22:08:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:44.896137 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.012629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01faaf8c-024a-414f-8dad-5bb3fe54c414 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.037318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:45.043299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708f52-241b-47bd-af1f-e6aa9f8bae1b None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.044241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708f52-241b-47bd-af1f-e6aa9f8bae1b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.044241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708f52-241b-47bd-af1f-e6aa9f8bae1b None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.045813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 469/941] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:45.050838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128334f-7873-4d88-8b25-72225dec9e29 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.051097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128334f-7873-4d88-8b25-72225dec9e29 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.051367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128334f-7873-4d88-8b25-72225dec9e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.053251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.054146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.054574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.055007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.066684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.068476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.072768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-b128334f-7873-4d88-8b25-72225dec9e29 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:45.074034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b128334f-7873-4d88-8b25-72225dec9e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:45.087886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:45.089009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:45.090167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:45.090167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:45.090167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:45.090167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:45.090167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 473/942] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:45.095558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.173203 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 470/943] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:45 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:45.202829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128334f-7873-4d88-8b25-72225dec9e29 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.226694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d11b65-a880-4549-a6bf-16ddb1fd4597 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.230084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:45.272669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.273325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.273798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.274198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.274901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:45.275206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.276955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.291349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 474/944] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:45.302484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.336202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128334f-7873-4d88-8b25-72225dec9e29 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.344718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:45.379442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.380090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.380365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.380631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.381268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.382778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:45.392105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:45.394810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:45.396835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:45.398940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 471/945] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:45.401650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.424500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176ce9b-e750-4133-88d7-b2493c5a739b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.464697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.465454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.465856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.466232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.475489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.477182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:45.492129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:45.493325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:45.494406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:45.499804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 475/946] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:08:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:45.500745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.509687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:45.528731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54274ef-825a-47a5-a7e2-eba8384f566b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.567093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.568039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.568501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.569048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.570134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:45.570625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.572227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.581798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 472/947] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:45 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:45.584244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.615310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:45.622223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b598cbe-1bc2-455d-a903-4545afb666f1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.659177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.659792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.660072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.660344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.661297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:45.661562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.663120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.687291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 476/948] 10.222.0.109 () {56 vars in 1154 bytes} [Thu Aug 4 22:08:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:45.688239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.699225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c5690-7970-4888-ac50-d3d14ad9909a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.737696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.738701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.739197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.739686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.740561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:45.741070 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.742668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.753141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 473/949] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:45 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:45.754383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.770135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:45.799574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a8bc8-7b3b-432c-b650-0027ef4e6f86 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.836982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0624814-1556-4c8f-ac4c-1f1bf1d6dc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.837672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624814-1556-4c8f-ac4c-1f1bf1d6dc13 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.837952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624814-1556-4c8f-ac4c-1f1bf1d6dc13 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.838223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624814-1556-4c8f-ac4c-1f1bf1d6dc13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.849807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0624814-1556-4c8f-ac4c-1f1bf1d6dc13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.851100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0624814-1556-4c8f-ac4c-1f1bf1d6dc13 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.862172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf7d24-2daa-48da-bf83-d00c71a6c48a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-b0624814-1556-4c8f-ac4c-1f1bf1d6dc13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:45.865986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:45.866881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:45.867751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:45.867751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:45.867751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:45.867751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:45.870532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 477/950] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:45.871662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd269b-6b61-41bd-9c93-e92bc9e6f3a6 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.873795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd269b-6b61-41bd-9c93-e92bc9e6f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.874081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd269b-6b61-41bd-9c93-e92bc9e6f3a6 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.875420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 478/951] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:45 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:45.876121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd269b-6b61-41bd-9c93-e92bc9e6f3a6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:45.881747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:45.906296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5e938-5e2f-457b-84b7-e7c910c8436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:45.907508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5e938-5e2f-457b-84b7-e7c910c8436d None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.908846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5e938-5e2f-457b-84b7-e7c910c8436d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.909455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5e938-5e2f-457b-84b7-e7c910c8436d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.910750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5e938-5e2f-457b-84b7-e7c910c8436d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:45.912636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5e938-5e2f-457b-84b7-e7c910c8436d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:45.937123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 474/952] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:08:45 2022] POST /identity/v3/endpoints => generated 351 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:45.939080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de3f66-febf-44c6-9070-88aab2fe3602 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:45.939555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de3f66-febf-44c6-9070-88aab2fe3602 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:45.940244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de3f66-febf-44c6-9070-88aab2fe3602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:45.967018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-38de3f66-febf-44c6-9070-88aab2fe3602 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:45.969062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-38de3f66-febf-44c6-9070-88aab2fe3602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:45.994434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd269b-6b61-41bd-9c93-e92bc9e6f3a6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:46.036547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:46.037185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:46.037491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:46.037766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:46.038470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:46.038821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:46.040225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:46.057945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 479/953] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:46.058549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:46.059703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:46.061789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 475/954] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:45 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:46.084998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de3f66-febf-44c6-9070-88aab2fe3602 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:46.140190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:46.193226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a44b86-284e-4dde-b871-9c23308795ef None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:46.229475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de3f66-febf-44c6-9070-88aab2fe3602 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:46.238300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:46.239292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:46.239775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:46.240263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:46.248200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:46.250101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:46.264188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:46.265202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:46.266229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:46.266229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:46.266229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:46.266229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:46.266229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:46.266229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:46.266229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:46.266229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:46.269912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 480/955] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:46.274838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:46.275504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:46.275840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:46.276361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:46.276774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] PATH_INFO: `/v3/services/compute` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:46.277586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:46.279159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:46.289580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:46.290841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:46.291959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:46.294487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 476/956] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:46.305486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:46.371016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:46.391487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bd38c-3e99-4027-af5c-3122a3b06f9c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:46.412556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:46.416990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c0512-51d2-4b95-b054-d44f8e907c1f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:46.427906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:46.428621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:46.429056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:46.429225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:46.430162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:46.430542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:46.432026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:46.449743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 481/957] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:46.451398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:46.457187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:46.458086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:46.458541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:46.459010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:46.459865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:46.460340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:46.461926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:46.471978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 477/958] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:46 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:46.473098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:46.556123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb67da8-6707-43ac-be02-da6c5555b67a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:46.581157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7aa85-c37a-41ac-bec1-ec79c88641d8 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:46.592897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb040cc-6fcd-429b-a9f7-6dee783c10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:46.593727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb040cc-6fcd-429b-a9f7-6dee783c10d7 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:46.594288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb040cc-6fcd-429b-a9f7-6dee783c10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:46.594398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb040cc-6fcd-429b-a9f7-6dee783c10d7 None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/212a75e45f2b449398acbfb80338afb0/roles/a449c88b1754437d942b32f2ec39cb9b` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:46.596723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb040cc-6fcd-429b-a9f7-6dee783c10d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=212a75e45f2b449398acbfb80338afb0, role_id=a449c88b1754437d942b32f2ec39cb9b)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:46.598476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb040cc-6fcd-429b-a9f7-6dee783c10d7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:46.610863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:46.619641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 482/959] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:08:46 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/212a75e45f2b449398acbfb80338afb0/roles/a449c88b1754437d942b32f2ec39cb9b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:08:46.621878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b30f7-fffc-439c-84e0-f1ee2e3ee5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:46.622747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b30f7-fffc-439c-84e0-f1ee2e3ee5ba None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:46.623299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b30f7-fffc-439c-84e0-f1ee2e3ee5ba None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:46.623784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b30f7-fffc-439c-84e0-f1ee2e3ee5ba None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:46.624714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b30f7-fffc-439c-84e0-f1ee2e3ee5ba None admin] RBAC: Adding query filter params (type=compute) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:46.625220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b30f7-fffc-439c-84e0-f1ee2e3ee5ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:46.631488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b30f7-fffc-439c-84e0-f1ee2e3ee5ba None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:46.637559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 478/960] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:46 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:47.144430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9ca8b-8c29-4ad2-9af4-076cdef66d71 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:47.144796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9ca8b-8c29-4ad2-9af4-076cdef66d71 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:47.144992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9ca8b-8c29-4ad2-9af4-076cdef66d71 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:47.146812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 483/961] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:47.152477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51206389-10cf-4a4b-a2a3-27c3cd39f103 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:47.152959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51206389-10cf-4a4b-a2a3-27c3cd39f103 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:47.153048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51206389-10cf-4a4b-a2a3-27c3cd39f103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:47.172404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-51206389-10cf-4a4b-a2a3-27c3cd39f103 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:47.173634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-51206389-10cf-4a4b-a2a3-27c3cd39f103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:47.246640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 479/962] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:47 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:47.271496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9ca8b-8c29-4ad2-9af4-076cdef66d71 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:47.319882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:47.395310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9ca8b-8c29-4ad2-9af4-076cdef66d71 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:47.434390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:47.435012 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:47.435289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:47.435559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:47.436233 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:47.437697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:47.455326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.sql.core [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 22:08:47.455326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 22:08:47.455326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [parameters: {'user_id': '275b5baa51594d4a8d81177bc1a46202', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 22:08:47.455326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91701) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application conn.query(q) Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result.read() Aug 04 22:08:47.462524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 22:08:47.463434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application next(self.gen) Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application yield resource Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application next(self.gen) Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise self.value Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application session.commit() Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 22:08:47.464410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.session.flush() Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application compat.raise_( Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 22:08:47.466049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 22:08:47.467240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application conn.query(q) Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application result.read() Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 22:08:47.468298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [parameters: {'user_id': '275b5baa51594d4a8d81177bc1a46202', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:47.469565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 22:08:47.470737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 22:08:47.474945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 22:08:47.474945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 22:08:47.474945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 22:08:47.474945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 22:08:47.474945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 22:08:47.474945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:47.474945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 484/963] 10.222.0.109 () {60 vars in 1136 bytes} [Thu Aug 4 22:08:47 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:47.475872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51206389-10cf-4a4b-a2a3-27c3cd39f103 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:47.576836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51206389-10cf-4a4b-a2a3-27c3cd39f103 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:47.597617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:47.612551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea6308-4cbc-42bb-b2ac-f0d869d58c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:47.613095 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea6308-4cbc-42bb-b2ac-f0d869d58c80 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:47.613378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea6308-4cbc-42bb-b2ac-f0d869d58c80 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:47.613683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea6308-4cbc-42bb-b2ac-f0d869d58c80 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:47.621500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea6308-4cbc-42bb-b2ac-f0d869d58c80 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:47.623116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea6308-4cbc-42bb-b2ac-f0d869d58c80 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-3aea6308-4cbc-42bb-b2ac-f0d869d58c80 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:47.636797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:47.637828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:47.638932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 480/964] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:47.651059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:47.763375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5db218-e75c-48fd-b4e4-0ab872df5e71 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:47.765402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:47.806769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5da934-8be5-4cc5-98c9-91a7d76e79a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:47.807389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5da934-8be5-4cc5-98c9-91a7d76e79a3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:47.807665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5da934-8be5-4cc5-98c9-91a7d76e79a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:47.807931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5da934-8be5-4cc5-98c9-91a7d76e79a3 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:47.808690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5da934-8be5-4cc5-98c9-91a7d76e79a3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:47.808987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5da934-8be5-4cc5-98c9-91a7d76e79a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:47.810879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5da934-8be5-4cc5-98c9-91a7d76e79a3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:47.851751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 485/965] 10.222.0.109 () {56 vars in 1106 bytes} [Thu Aug 4 22:08:47 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:48.062365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83e81b-475f-4824-adfb-ebfc09c3b083 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.062717 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83e81b-475f-4824-adfb-ebfc09c3b083 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.062838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83e81b-475f-4824-adfb-ebfc09c3b083 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.064827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 481/966] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:48.071144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d5049-8a1a-417f-9709-df0fae49332c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.071674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d5049-8a1a-417f-9709-df0fae49332c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.072164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d5049-8a1a-417f-9709-df0fae49332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.093599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-df2d5049-8a1a-417f-9709-df0fae49332c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:48.095399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-df2d5049-8a1a-417f-9709-df0fae49332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:48.172246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 486/967] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:48 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:48.195951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83e81b-475f-4824-adfb-ebfc09c3b083 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:48.300244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83e81b-475f-4824-adfb-ebfc09c3b083 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:48.334603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:48.335254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.335537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.335810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.336434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:48.337845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:48.347314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:48.348311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:48.349157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 482/968] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:48.356339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d5049-8a1a-417f-9709-df0fae49332c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:48.448137 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:48.459073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d5049-8a1a-417f-9709-df0fae49332c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:48.494322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:48.494944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.495224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.495495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.496166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:48.496470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:48.497887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:48.507396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 487/969] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:48 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:48.508533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:48.556744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:48.613456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099d49e-1b76-4ca1-a981-7baaab139194 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:48.647650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a443ad6-7663-4713-80b4-f8de01103301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:48.648274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443ad6-7663-4713-80b4-f8de01103301 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.648748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443ad6-7663-4713-80b4-f8de01103301 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.649035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443ad6-7663-4713-80b4-f8de01103301 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.649753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a443ad6-7663-4713-80b4-f8de01103301 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:48.650059 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a443ad6-7663-4713-80b4-f8de01103301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:48.651427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a443ad6-7663-4713-80b4-f8de01103301 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:48.661452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 483/970] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:48 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:48.663133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:48.763971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c64fd-a685-4dcc-baab-b5bf011d2485 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:48.786632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2a50b-baf1-417f-a14d-cf3ab3855b17 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.786982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2a50b-baf1-417f-a14d-cf3ab3855b17 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.787182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2a50b-baf1-417f-a14d-cf3ab3855b17 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.788529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 484/971] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:48.791747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e442e-583b-4d82-b385-d005282cbad6 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.792196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e442e-583b-4d82-b385-d005282cbad6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.792323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e442e-583b-4d82-b385-d005282cbad6 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.793544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 485/972] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:48 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:48.794744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ef494-287c-4d23-aafa-44584c2f350d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.795036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ef494-287c-4d23-aafa-44584c2f350d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.795302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ef494-287c-4d23-aafa-44584c2f350d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.801539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97cef8-aca3-4881-817a-6197fefdee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:48.802160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97cef8-aca3-4881-817a-6197fefdee5a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.802441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97cef8-aca3-4881-817a-6197fefdee5a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.802709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97cef8-aca3-4881-817a-6197fefdee5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.803466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97cef8-aca3-4881-817a-6197fefdee5a None admin] RBAC: Adding query filter params (interface=admin, service_id=a5eefdf368ec4963ab4b8fcd8dc5994b, region_id=RegionOne) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:48.803771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97cef8-aca3-4881-817a-6197fefdee5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:48.805133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97cef8-aca3-4881-817a-6197fefdee5a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:48.810767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 488/973] 10.222.0.109 () {56 vars in 1254 bytes} [Thu Aug 4 22:08:48 2022] GET /identity/v3/endpoints?service_id=a5eefdf368ec4963ab4b8fcd8dc5994b&interface=admin®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:48.811876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059cc14-847c-4577-80be-ae2de41c9719 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:48.812162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059cc14-847c-4577-80be-ae2de41c9719 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:48.812448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059cc14-847c-4577-80be-ae2de41c9719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:48.814995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-752ef494-287c-4d23-aafa-44584c2f350d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:48.816586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-752ef494-287c-4d23-aafa-44584c2f350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:48.831414 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0059cc14-847c-4577-80be-ae2de41c9719 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:48.832585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0059cc14-847c-4577-80be-ae2de41c9719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:48.845440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:48.887941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 486/974] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:48 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:48.888790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ef494-287c-4d23-aafa-44584c2f350d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:48.903987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 489/975] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:48 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:48.914518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059cc14-847c-4577-80be-ae2de41c9719 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:48.933649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:48.994859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ef494-287c-4d23-aafa-44584c2f350d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:49.015957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059cc14-847c-4577-80be-ae2de41c9719 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:49.029474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19265115-3611-44a2-9661-81493418b31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:49.030105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265115-3611-44a2-9661-81493418b31c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:49.030384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265115-3611-44a2-9661-81493418b31c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:49.030654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265115-3611-44a2-9661-81493418b31c None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:49.031301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19265115-3611-44a2-9661-81493418b31c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:49.032666 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19265115-3611-44a2-9661-81493418b31c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:49.039161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 487/976] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:08:48 2022] GET /identity/v3/services => generated 1504 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:08:49.039931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19265115-3611-44a2-9661-81493418b31c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:49.053609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9c0c2-ed01-4620-b84d-0a8a5e712ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:49.054284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9c0c2-ed01-4620-b84d-0a8a5e712ecc None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:49.054565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9c0c2-ed01-4620-b84d-0a8a5e712ecc None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:49.054849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9c0c2-ed01-4620-b84d-0a8a5e712ecc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:49.068363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9c0c2-ed01-4620-b84d-0a8a5e712ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-1af9c0c2-ed01-4620-b84d-0a8a5e712ecc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:49.073547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:49.074602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:08:49.075460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:49.080946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 490/977] 10.222.0.109 () {56 vars in 1125 bytes} [Thu Aug 4 22:08:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:49.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9c0c2-ed01-4620-b84d-0a8a5e712ecc None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:49.095162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:49.172401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19265115-3611-44a2-9661-81493418b31c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:49.175406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:49.201526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9c0c2-ed01-4620-b84d-0a8a5e712ecc None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:49.216422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd5f81-d0cb-48f8-947f-68fae34ebf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:49.217055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd5f81-d0cb-48f8-947f-68fae34ebf53 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:49.217338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd5f81-d0cb-48f8-947f-68fae34ebf53 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:49.217658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd5f81-d0cb-48f8-947f-68fae34ebf53 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:49.218353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd5f81-d0cb-48f8-947f-68fae34ebf53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:49.219715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd5f81-d0cb-48f8-947f-68fae34ebf53 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:49.237250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 488/978] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:08:49 2022] POST /identity/v3/services => generated 243 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:49.250771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:49.251610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:49.252560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:49.252966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:49.253909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:49.254377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:49.256061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:49.266521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 491/979] 10.222.0.109 () {56 vars in 1120 bytes} [Thu Aug 4 22:08:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:49.274035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd5f81-d0cb-48f8-947f-68fae34ebf53 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:49.387765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd5f81-d0cb-48f8-947f-68fae34ebf53 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:49.424934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:49.425530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:49.425814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:49.426087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:49.437132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:49.438960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:49.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:49.456220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:49.457127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:49.457127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:49.457127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:49.457127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:49.457127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 489/980] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:49.463171 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:49.532312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:49.568960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9833ff8f-c37e-4b1b-8fc2-583f68b9e69e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:49.602638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:49.603262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:49.603539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:49.603809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:49.604502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:49.604804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:49.606205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:49.623317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 492/981] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:49.624622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:49.642720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:49.739345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54803a7-02f9-489b-832d-6483cc34bcd1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:49.775330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:49.775971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:49.776249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:49.776521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:49.785338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:49.786846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:49.798143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:49.799763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:49.800714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 490/982] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:08:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:49.807621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:49.909596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871a3d7-6d84-4cae-8085-bb8d2cbe71d4 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:49.935323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:49.945921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:49.946546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:49.946833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:49.947104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:49.947896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:49.948202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:49.949659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:49.971331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 493/983] 10.222.0.109 () {56 vars in 1154 bytes} [Thu Aug 4 22:08:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:49.974103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:50.058980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:50.089545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d854a-9adc-410b-836b-17380508f9f4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:50.132258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ccc93-40d3-4173-afd1-df61f164f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:50.132811 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ccc93-40d3-4173-afd1-df61f164f77d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:50.133094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ccc93-40d3-4173-afd1-df61f164f77d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:50.133378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ccc93-40d3-4173-afd1-df61f164f77d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:50.203704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ccc93-40d3-4173-afd1-df61f164f77d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:50.205288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ccc93-40d3-4173-afd1-df61f164f77d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:50.279048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:50.310770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 491/984] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:49 2022] GET /identity/v3/auth/tokens => generated 1572 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 22:08:50.311854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:50.418579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73232a2-1818-4e6d-8336-348d6f8f99c0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:50.453015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:50.453680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:50.453958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:50.454228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:50.464163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:50.465641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:50.480237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:50.481127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:50.487682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:50.487682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:50.487682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:50.487682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:50.487682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 494/985] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:50.489680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ccc93-40d3-4173-afd1-df61f164f77d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:50.594225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ccc93-40d3-4173-afd1-df61f164f77d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:50.630584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:50.631207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:50.631483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:50.631754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:50.632443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:50.632741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:50.634126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:50.646786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 492/986] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:50.652051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:50.690654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:50.760637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113dec3-6700-4e0f-9439-9ccfabf5f1c9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:50.795810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:50.796449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:50.796727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:50.797136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:50.804661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:50.806158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:50.817482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:50.823622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:50.824563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 495/987] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:50.825064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:50.860223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:50.928499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aae80-43dc-4990-b01d-d7b370d344d9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:50.963193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc9606-5d9d-4d8c-a882-4c2c624b9e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:50.963815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9606-5d9d-4d8c-a882-4c2c624b9e9c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:50.964096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9606-5d9d-4d8c-a882-4c2c624b9e9c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:50.964370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9606-5d9d-4d8c-a882-4c2c624b9e9c None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:50.965116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc9606-5d9d-4d8c-a882-4c2c624b9e9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:50.965444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc9606-5d9d-4d8c-a882-4c2c624b9e9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:50.966866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc9606-5d9d-4d8c-a882-4c2c624b9e9c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:50.979149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 493/988] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:50.984896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.084248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:51.091293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5c13c-5529-49d3-bae7-4f9e6defa629 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:51.104193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64d42e-14b6-4c39-a608-f92ac2825ce3 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.104484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64d42e-14b6-4c39-a608-f92ac2825ce3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.104754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64d42e-14b6-4c39-a608-f92ac2825ce3 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.107437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 494/989] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:51.115028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e25cb-8831-4d5e-9974-e370527fa580 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.115314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e25cb-8831-4d5e-9974-e370527fa580 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.115586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e25cb-8831-4d5e-9974-e370527fa580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.133459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:51.134098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.134443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.134724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.134842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ff0e25cb-8831-4d5e-9974-e370527fa580 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:51.135541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] RBAC: Adding query filter params (role.id=a449c88b1754437d942b32f2ec39cb9b, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=212a75e45f2b449398acbfb80338afb0) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:51.135852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:51.136197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ff0e25cb-8831-4d5e-9974-e370527fa580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:51.137421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:51.143931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 496/990] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:50 2022] GET /identity/v3/role_assignments?user.id=212a75e45f2b449398acbfb80338afb0&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=a449c88b1754437d942b32f2ec39cb9b => generated 610 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:51.215921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 495/991] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:51 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:51.240686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.264822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1cd6-0b01-4925-80d6-2786d79e0e45 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.265157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1cd6-0b01-4925-80d6-2786d79e0e45 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.265453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1cd6-0b01-4925-80d6-2786d79e0e45 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.267916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 496/992] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:51.273693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5cac2-aeb6-4eb8-b31b-663c438b417b None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.273988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5cac2-aeb6-4eb8-b31b-663c438b417b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.274255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5cac2-aeb6-4eb8-b31b-663c438b417b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.293369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-53a5cac2-aeb6-4eb8-b31b-663c438b417b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:51.294615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-53a5cac2-aeb6-4eb8-b31b-663c438b417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:51.317646 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:51.365255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e29283-3fcb-4952-bbee-9407affe9b43 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:51.373453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 497/993] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:51 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:51.398718 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5cac2-aeb6-4eb8-b31b-663c438b417b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.404824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:51.405508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.405789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.406059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.406682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:51.409227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:51.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:51.419500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 22:08:51.420347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:51.425161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 497/994] 10.222.0.109 () {56 vars in 1131 bytes} [Thu Aug 4 22:08:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:51.426067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.454049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:51.508410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5cac2-aeb6-4eb8-b31b-663c438b417b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:51.534988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b09db9-56f6-4ad5-8f8e-2223a07f4bc8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:51.545840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:51.546490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.546772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.547046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.547673 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:51.549077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:51.558056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:51.559036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:51.559907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 498/995] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:51.565852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.574730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:51.575351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.575664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.575936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.576617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:51.576916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:51.578376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:51.584628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 498/996] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:51 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:51.590215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.592989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:51.687680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:51.694270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad9f9d-021b-4785-848d-cffa6c81f4f5 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:51.702872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912bd19-e503-4f8f-bac3-67d10c02a06a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:51.728953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:51.729613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.729903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.730178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.730891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] RBAC: Adding query filter params (name=compute) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:51.731167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:51.732732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:51.740932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:51.741874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.742315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 499/997] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:51 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:51.742430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.742835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.743604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:51.743916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:51.745354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:51.747840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.751846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 499/998] 10.222.0.109 () {56 vars in 1127 bytes} [Thu Aug 4 22:08:51 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:51.758142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.836585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:51.855375 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d2bc9-1034-43a5-8868-f6084a74351a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:51.867345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd65fb-2805-4f20-b0b3-cfdded96150d None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:51.893225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:51.893914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.894189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.894458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.895127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] RBAC: Adding query filter params (type=compute) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:51.895430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:51.896791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:51.901902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 500/999] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:51 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:51.904109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574ca75-595d-48ae-87f2-371659d553ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:51.904734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574ca75-595d-48ae-87f2-371659d553ce None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:51.905088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574ca75-595d-48ae-87f2-371659d553ce None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:51.905449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574ca75-595d-48ae-87f2-371659d553ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:51.906199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574ca75-595d-48ae-87f2-371659d553ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:51.907472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:51.907597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574ca75-595d-48ae-87f2-371659d553ce None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:51.938105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 500/1000] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:08:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:08:52.021422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10781159-4f30-45ac-bf1b-a05dc2e28b1d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:52.063593 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176dd85-2f8e-48b7-9fe1-cf98abf4a2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:52.064446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176dd85-2f8e-48b7-9fe1-cf98abf4a2d8 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:52.064741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176dd85-2f8e-48b7-9fe1-cf98abf4a2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:52.065015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176dd85-2f8e-48b7-9fe1-cf98abf4a2d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:52.065834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176dd85-2f8e-48b7-9fe1-cf98abf4a2d8 None admin] RBAC: Adding query filter params (interface=public, service_id=5670c5c0d6aa436384b0fc55292b30fb, region_id=RegionOne) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:52.066145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176dd85-2f8e-48b7-9fe1-cf98abf4a2d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:52.067535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176dd85-2f8e-48b7-9fe1-cf98abf4a2d8 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:52.073441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 501/1001] 10.222.0.109 () {56 vars in 1256 bytes} [Thu Aug 4 22:08:51 2022] GET /identity/v3/endpoints?service_id=5670c5c0d6aa436384b0fc55292b30fb&interface=public®ion_id=RegionOne => generated 195 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:52.078880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574ca75-595d-48ae-87f2-371659d553ce None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:52.098528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:52.188455 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574ca75-595d-48ae-87f2-371659d553ce None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:52.222451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50c9c9-4d7d-4f20-bde1-1ce8c113a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:52.223076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c9c9-4d7d-4f20-bde1-1ce8c113a57a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:52.223356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c9c9-4d7d-4f20-bde1-1ce8c113a57a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:52.223635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c9c9-4d7d-4f20-bde1-1ce8c113a57a None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:52.224290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50c9c9-4d7d-4f20-bde1-1ce8c113a57a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:52.225706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50c9c9-4d7d-4f20-bde1-1ce8c113a57a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:52.232029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 501/1002] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:08:52 2022] GET /identity/v3/services => generated 1735 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:08:53.191088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00bdf6-65e4-43e6-9ba3-4817332a7e5b None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:53.191641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00bdf6-65e4-43e6-9ba3-4817332a7e5b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:53.191641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00bdf6-65e4-43e6-9ba3-4817332a7e5b None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:53.192990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 502/1003] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:53.198230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8400197-abe1-48d5-a3e2-15a7c2413cd6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:53.198651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8400197-abe1-48d5-a3e2-15a7c2413cd6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:53.198815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8400197-abe1-48d5-a3e2-15a7c2413cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:53.217020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b8400197-abe1-48d5-a3e2-15a7c2413cd6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:53.218221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b8400197-abe1-48d5-a3e2-15a7c2413cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:53.289209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 502/1004] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:53 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:53.311848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00bdf6-65e4-43e6-9ba3-4817332a7e5b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:53.321842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:53.422729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00bdf6-65e4-43e6-9ba3-4817332a7e5b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:53.458233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:53.458873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:53.459156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:53.459427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] PATH_INFO: `/v3/services/image` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:53.460044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:53.461438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:53.470351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:53.471333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:08:53.472156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:53.476962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 503/1005] 10.222.0.109 () {56 vars in 1110 bytes} [Thu Aug 4 22:08:53 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:53.477877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8400197-abe1-48d5-a3e2-15a7c2413cd6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:53.486921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:53.582118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8400197-abe1-48d5-a3e2-15a7c2413cd6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:53.617330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343d99a-33b0-4a53-9c28-82f0ed107488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:53.617981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343d99a-33b0-4a53-9c28-82f0ed107488 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:53.618261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343d99a-33b0-4a53-9c28-82f0ed107488 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:53.618537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343d99a-33b0-4a53-9c28-82f0ed107488 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:53.619267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343d99a-33b0-4a53-9c28-82f0ed107488 None admin] RBAC: Adding query filter params (name=image) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:53.619571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343d99a-33b0-4a53-9c28-82f0ed107488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:53.620946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343d99a-33b0-4a53-9c28-82f0ed107488 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:53.630919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 503/1006] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:53 2022] GET /identity/v3/services?name=image => generated 123 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:53.632424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:53.707119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:53.741007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d532259-785f-421b-93bd-123723c5d58b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:53.777042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af826884-3195-4484-a578-53309df942bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:53.777696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af826884-3195-4484-a578-53309df942bf None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:53.777976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af826884-3195-4484-a578-53309df942bf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:53.778244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af826884-3195-4484-a578-53309df942bf None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:53.778913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af826884-3195-4484-a578-53309df942bf None admin] RBAC: Adding query filter params (type=image) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:53.779210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af826884-3195-4484-a578-53309df942bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:53.780627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af826884-3195-4484-a578-53309df942bf None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:53.793441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 504/1007] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:53 2022] GET /identity/v3/services?type=image => generated 123 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:54.004473 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b551bf-0b75-423d-9f27-d7a6d76c49cb None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.004711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b551bf-0b75-423d-9f27-d7a6d76c49cb None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.004944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b551bf-0b75-423d-9f27-d7a6d76c49cb None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.006371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 504/1008] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:54.011637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6167-fb8a-4281-856e-0d6b876f4f20 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.011928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6167-fb8a-4281-856e-0d6b876f4f20 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.012204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6167-fb8a-4281-856e-0d6b876f4f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.031300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-728a6167-fb8a-4281-856e-0d6b876f4f20 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:54.032495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-728a6167-fb8a-4281-856e-0d6b876f4f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:54.103734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 505/1009] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:54 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:54.128994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b551bf-0b75-423d-9f27-d7a6d76c49cb None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:54.203041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:54.239242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b551bf-0b75-423d-9f27-d7a6d76c49cb None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:54.266252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72501f3a-5f38-42a4-9c74-18bf7f99d822 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.267362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72501f3a-5f38-42a4-9c74-18bf7f99d822 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.267687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72501f3a-5f38-42a4-9c74-18bf7f99d822 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.269096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 506/1010] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:54 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:54.274362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e07f4b-066c-4c33-999d-87f76ada359d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.274647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e07f4b-066c-4c33-999d-87f76ada359d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.274919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e07f4b-066c-4c33-999d-87f76ada359d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.276360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1940e8-dc6a-46b7-b855-3826c32b2003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:54.277015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1940e8-dc6a-46b7-b855-3826c32b2003 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.277299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1940e8-dc6a-46b7-b855-3826c32b2003 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.277712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1940e8-dc6a-46b7-b855-3826c32b2003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.278920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1940e8-dc6a-46b7-b855-3826c32b2003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:54.280413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1940e8-dc6a-46b7-b855-3826c32b2003 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:54.288063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 505/1011] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:54 2022] GET /identity/v3/domains/default => generated 200 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:54.289096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1940e8-dc6a-46b7-b855-3826c32b2003 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:54.294661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-42e07f4b-066c-4c33-999d-87f76ada359d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:54.295857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-42e07f4b-066c-4c33-999d-87f76ada359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:54.360455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:54.373469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 507/1012] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:54 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:54.394283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e07f4b-066c-4c33-999d-87f76ada359d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:54.399661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1940e8-dc6a-46b7-b855-3826c32b2003 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:54.436356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a0f36-4357-4d72-a9a4-ff2752dcece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:54.436974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0f36-4357-4d72-a9a4-ff2752dcece1 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.437257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0f36-4357-4d72-a9a4-ff2752dcece1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.437564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0f36-4357-4d72-a9a4-ff2752dcece1 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.438262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a0f36-4357-4d72-a9a4-ff2752dcece1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:54.440321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a0f36-4357-4d72-a9a4-ff2752dcece1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:54.450842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:54.472149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 506/1013] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:08:54 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:08:54.504027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e07f4b-066c-4c33-999d-87f76ada359d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:54.544529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:54.545253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.545557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.545822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] PATH_INFO: `/v3/services/compute` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.546442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:54.547820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:54.558533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:54.559618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 22:08:54.560465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:54.561895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 508/1014] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:54.566165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a0f36-4357-4d72-a9a4-ff2752dcece1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:54.677028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a0f36-4357-4d72-a9a4-ff2752dcece1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:54.716394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:54.717073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.717357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.717677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.718356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] RBAC: Adding query filter params (name=compute) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:54.718797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:54.720739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:54.726083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 507/1015] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:54 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:54.733133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:54.794722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:54.839762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f6e6-0485-4cd6-8af7-95e07837848a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:54.874383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030be9a2-3b63-477c-aff8-78cd5dccc0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:54.875009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be9a2-3b63-477c-aff8-78cd5dccc0aa None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:54.875285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be9a2-3b63-477c-aff8-78cd5dccc0aa None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:54.875555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be9a2-3b63-477c-aff8-78cd5dccc0aa None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:54.876224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be9a2-3b63-477c-aff8-78cd5dccc0aa None admin] RBAC: Adding query filter params (type=compute) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:54.876624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be9a2-3b63-477c-aff8-78cd5dccc0aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:54.878029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be9a2-3b63-477c-aff8-78cd5dccc0aa None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:54.883314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 509/1016] 10.222.0.109 () {56 vars in 1117 bytes} [Thu Aug 4 22:08:54 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:54.888932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:54.950678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:54.998441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85a16-9cce-44c6-96bd-bb853a33bfda None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:55.033340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590eea83-5c92-4ab1-99f2-2dc38ca25093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:55.034221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eea83-5c92-4ab1-99f2-2dc38ca25093 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:55.034648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eea83-5c92-4ab1-99f2-2dc38ca25093 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:55.035092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eea83-5c92-4ab1-99f2-2dc38ca25093 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:55.035969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590eea83-5c92-4ab1-99f2-2dc38ca25093 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:55.037521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590eea83-5c92-4ab1-99f2-2dc38ca25093 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:55.065575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 508/1017] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:08:54 2022] POST /identity/v3/endpoints => generated 336 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:08:55.816079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143029-bccc-4b1c-b231-385949f06317 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:55.816633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143029-bccc-4b1c-b231-385949f06317 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:55.816633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143029-bccc-4b1c-b231-385949f06317 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:55.818160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 510/1018] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:55.823164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6400a-7849-44f1-aeac-cfa969f65c98 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:55.823549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6400a-7849-44f1-aeac-cfa969f65c98 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:55.823711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6400a-7849-44f1-aeac-cfa969f65c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:55.842013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f3a6400a-7849-44f1-aeac-cfa969f65c98 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:55.843205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f3a6400a-7849-44f1-aeac-cfa969f65c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:55.919834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 509/1019] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:55 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:55.942983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40143029-bccc-4b1c-b231-385949f06317 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:56.046685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40143029-bccc-4b1c-b231-385949f06317 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:56.069730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:56.082322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3c05e-d277-4a16-9230-1fdafe1e0330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:56.082944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3c05e-d277-4a16-9230-1fdafe1e0330 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:56.083227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3c05e-d277-4a16-9230-1fdafe1e0330 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:56.083501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3c05e-d277-4a16-9230-1fdafe1e0330 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:56.084202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3c05e-d277-4a16-9230-1fdafe1e0330 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:56.085675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3c05e-d277-4a16-9230-1fdafe1e0330 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:56.113438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 511/1020] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:08:55 2022] POST /identity/v3/services => generated 243 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:56.558422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707427b9-ef34-4bd8-9677-16a8c88de7aa None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:56.558712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707427b9-ef34-4bd8-9677-16a8c88de7aa None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:56.559170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707427b9-ef34-4bd8-9677-16a8c88de7aa None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:56.560587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 510/1021] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:56.566578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14134c-f35e-4aea-9048-cb6b9dded607 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:56.566869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14134c-f35e-4aea-9048-cb6b9dded607 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:56.567142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14134c-f35e-4aea-9048-cb6b9dded607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:56.585857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9d14134c-f35e-4aea-9048-cb6b9dded607 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:56.587042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9d14134c-f35e-4aea-9048-cb6b9dded607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:56.662100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 512/1022] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:56 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:56.685871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707427b9-ef34-4bd8-9677-16a8c88de7aa None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:56.724047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:56.794033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707427b9-ef34-4bd8-9677-16a8c88de7aa None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:56.828901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b339cb-ba2e-4dd8-870e-f20754249d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:56.830237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b339cb-ba2e-4dd8-870e-f20754249d07 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:56.830319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b339cb-ba2e-4dd8-870e-f20754249d07 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:56.830595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b339cb-ba2e-4dd8-870e-f20754249d07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:56.831754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b339cb-ba2e-4dd8-870e-f20754249d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:56.833230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b339cb-ba2e-4dd8-870e-f20754249d07 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:56.839435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 511/1023] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:56 2022] GET /identity/v3/domains/default => generated 200 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:56.841113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14134c-f35e-4aea-9048-cb6b9dded607 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:56.940359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14134c-f35e-4aea-9048-cb6b9dded607 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:56.949793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:56.977189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d2c96-d33d-4dbe-832d-350c0d072d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:56.978087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d2c96-d33d-4dbe-832d-350c0d072d36 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:56.978510 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d2c96-d33d-4dbe-832d-350c0d072d36 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:56.978947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d2c96-d33d-4dbe-832d-350c0d072d36 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:56.979770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d2c96-d33d-4dbe-832d-350c0d072d36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:56.981341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d2c96-d33d-4dbe-832d-350c0d072d36 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:57.009457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 513/1024] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:08:56 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:08:57.195934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae01c-d4da-4a95-aa21-f060d39c375f None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:57.196216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae01c-d4da-4a95-aa21-f060d39c375f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:57.196488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae01c-d4da-4a95-aa21-f060d39c375f None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:57.198051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 512/1025] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:57.202965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbc7c9-b58e-4062-862d-50d1fb95d071 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:57.203258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbc7c9-b58e-4062-862d-50d1fb95d071 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:57.203535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbc7c9-b58e-4062-862d-50d1fb95d071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:57.223166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-20fbc7c9-b58e-4062-862d-50d1fb95d071 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:57.224396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-20fbc7c9-b58e-4062-862d-50d1fb95d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:57.300618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 514/1026] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:57 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:08:57.324384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aae01c-d4da-4a95-aa21-f060d39c375f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:57.426981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aae01c-d4da-4a95-aa21-f060d39c375f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:57.429198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:57.462516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c682f2-f76f-4d99-a1a2-a24165d35f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:57.463151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c682f2-f76f-4d99-a1a2-a24165d35f50 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:57.463431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c682f2-f76f-4d99-a1a2-a24165d35f50 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:57.463702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c682f2-f76f-4d99-a1a2-a24165d35f50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:57.474697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c682f2-f76f-4d99-a1a2-a24165d35f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a2c682f2-f76f-4d99-a1a2-a24165d35f50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:57.479592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:57.480534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:08:57.481402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:57.482945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 513/1027] 10.222.0.109 () {56 vars in 1125 bytes} [Thu Aug 4 22:08:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:57.488587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbc7c9-b58e-4062-862d-50d1fb95d071 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:57.585729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:57.594007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbc7c9-b58e-4062-862d-50d1fb95d071 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:57.630321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:57.631032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:57.631313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:57.631584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:57.632302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:57.632605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:57.634006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:57.647281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 515/1028] 10.222.0.109 () {56 vars in 1120 bytes} [Thu Aug 4 22:08:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:57.648802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c682f2-f76f-4d99-a1a2-a24165d35f50 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:57.754012 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c682f2-f76f-4d99-a1a2-a24165d35f50 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:57.788862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:57.789538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:57.789827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:57.790102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:57.800770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:57.802261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:57.817927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:57.818897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:57.819805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:57.819805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:57.819805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:57.819805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:57.819805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 514/1029] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:57.826069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:57.930171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ea2fb-b0b3-4e30-93c4-07de70a0dfca None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:57.964625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a0910-ddb8-4832-9d2e-e609ff871d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:57.965238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a0910-ddb8-4832-9d2e-e609ff871d78 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:57.965545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a0910-ddb8-4832-9d2e-e609ff871d78 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:57.965816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a0910-ddb8-4832-9d2e-e609ff871d78 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:57.966499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a0910-ddb8-4832-9d2e-e609ff871d78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:57.966798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a0910-ddb8-4832-9d2e-e609ff871d78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:57.968154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a0910-ddb8-4832-9d2e-e609ff871d78 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:57.980563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 516/1030] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:57.986988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.006782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:58.091883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e02755-3b42-49fe-b240-745d5edce2f8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:58.131877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:58.132524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.132806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.133079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.141028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:58.142959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:58.145629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fb02f-6382-4409-85c2-c11fc7318a2d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.146015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fb02f-6382-4409-85c2-c11fc7318a2d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.146349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fb02f-6382-4409-85c2-c11fc7318a2d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.147932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 517/1031] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:08:58.152704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a6451-91a5-4f7d-bfb5-f9752de8de7a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.152992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a6451-91a5-4f7d-bfb5-f9752de8de7a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.153267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a6451-91a5-4f7d-bfb5-f9752de8de7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:08:58.158765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:58.159688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:58.161218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 515/1032] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:08:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:58.165503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.171618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-209a6451-91a5-4f7d-bfb5-f9752de8de7a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:58.172819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-209a6451-91a5-4f7d-bfb5-f9752de8de7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:58.185128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:58.253489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 518/1033] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:58 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:58.256594 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:58.274204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a6451-91a5-4f7d-bfb5-f9752de8de7a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.282314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc49a8-765d-4ccd-b057-0c5866d674d1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:58.318440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:58.319062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.319644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.319928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.320716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:58.321020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:58.322540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:58.341832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 516/1034] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:58.347732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.386964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a6451-91a5-4f7d-bfb5-f9752de8de7a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:58.424986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:58.425665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.425951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.426222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] PATH_INFO: `/v3/services/image` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.427211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:58.428896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:58.437949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:58.439216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:08:58.440065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 519/1035] 10.222.0.109 () {56 vars in 1110 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:58.445001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.453358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1720c-feb0-4849-8570-234dab81bba5 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:58.457647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:58.490984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d270ac0-82e4-4b8b-830c-77fd3490989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:58.491606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d270ac0-82e4-4b8b-830c-77fd3490989f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.491889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d270ac0-82e4-4b8b-830c-77fd3490989f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.492163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d270ac0-82e4-4b8b-830c-77fd3490989f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.494827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:58.562050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711708d-5289-4ef8-aecc-bbd6523170c6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:58.565072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d270ac0-82e4-4b8b-830c-77fd3490989f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:58.566634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d270ac0-82e4-4b8b-830c-77fd3490989f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:58.603665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:58.604080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.604364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.604634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.605314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] RBAC: Adding query filter params (name=image) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:58.605643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:58.607005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:58.618312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 520/1036] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/services?name=image => generated 123 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:58.619329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.679635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 517/1037] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 22:08:58.685039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d270ac0-82e4-4b8b-830c-77fd3490989f None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.702532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:58.720876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:58.731712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843529-416b-4c01-a5d8-512a8c585dad None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:58.775591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:58.776249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.776527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.776798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.777524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] RBAC: Adding query filter params (type=image) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:58.777830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:58.779209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:58.789737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 521/1038] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/services?type=image => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:58.790938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.824306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d270ac0-82e4-4b8b-830c-77fd3490989f None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:58.862535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:58.863100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.863381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.863657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.873912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:58.875365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:08:58.892453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:58.893465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:58.894350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:08:58.894350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:08:58.894350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:08:58.894350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:58.898516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 518/1039] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:58.899402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.923543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d54c4-9366-42f8-a124-8cf9015b75e9 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:58.960489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:58.962111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:58.962640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:58.963204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:58.964171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] RBAC: Adding query filter params (interface=public, service_id=0465534e4723452e8e529fcb1c30063f, region_id=RegionOne) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:58.964769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:58.966430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:58.977739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 522/1040] 10.222.0.109 () {56 vars in 1256 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/endpoints?service_id=0465534e4723452e8e529fcb1c30063f&interface=public®ion_id=RegionOne => generated 195 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:58.978898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:58.981900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:58.999909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:59.017732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd975-a429-48f6-ac9b-e1628de77c81 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.066934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.067631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.068333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.068613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.069319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:59.069657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:59.071753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:59.085229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 519/1041] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:59.090410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:59.107424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42951b-febe-468c-9813-58975912d8f7 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.151578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a3bf7-4aa9-4abc-b53f-03589abf2cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.152213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a3bf7-4aa9-4abc-b53f-03589abf2cf9 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.152497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a3bf7-4aa9-4abc-b53f-03589abf2cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.152774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a3bf7-4aa9-4abc-b53f-03589abf2cf9 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.153455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a3bf7-4aa9-4abc-b53f-03589abf2cf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:59.154857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a3bf7-4aa9-4abc-b53f-03589abf2cf9 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:59.163056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 523/1042] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:08:58 2022] GET /identity/v3/services => generated 1966 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:08:59.164341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75532932-5212-49f8-8206-a820d0c186b0 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.164624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75532932-5212-49f8-8206-a820d0c186b0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.164911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75532932-5212-49f8-8206-a820d0c186b0 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.166291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 524/1043] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:08:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:08:59.171586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4d8e3-2660-4d7f-9788-ca7505bbb470 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.171864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4d8e3-2660-4d7f-9788-ca7505bbb470 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.172132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4d8e3-2660-4d7f-9788-ca7505bbb470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.191810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-c0f4d8e3-2660-4d7f-9788-ca7505bbb470 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:08:59.193041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c0f4d8e3-2660-4d7f-9788-ca7505bbb470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:08:59.205548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd6443-3467-46b3-93e5-f024ad985640 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.247860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.248852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.249303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.249630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.259183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:59.260732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:59.261709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:59.270499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 525/1044] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:08:59 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:08:59.275993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:59.276991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:59.278111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 520/1045] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:08:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:08:59.283658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4d8e3-2660-4d7f-9788-ca7505bbb470 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:59.293073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:59.302638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:59.400398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4d8e3-2660-4d7f-9788-ca7505bbb470 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.406792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76ebef-2571-47c8-bfbf-feaeac3fbce1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.437432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.438176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.438498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.439401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.440159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:59.440518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:59.442159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:59.448173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a1ab-a3ce-4623-ba02-5c51a8d51b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.448813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a1ab-a3ce-4623-ba02-5c51a8d51b33 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.449576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a1ab-a3ce-4623-ba02-5c51a8d51b33 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.449852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a1ab-a3ce-4623-ba02-5c51a8d51b33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.454218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 526/1046] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:08:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:59.460002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:59.461017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1a1ab-a3ce-4623-ba02-5c51a8d51b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-d7d1a1ab-a3ce-4623-ba02-5c51a8d51b33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:08:59.466360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:08:59.467300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:08:59.468162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:08:59.473040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 521/1047] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:08:59.473581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a1ab-a3ce-4623-ba02-5c51a8d51b33 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:59.520209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:59.544793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:59.567804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d94f26-9031-46e5-bc70-32c4c54be905 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.583328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a1ab-a3ce-4623-ba02-5c51a8d51b33 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.601881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.602504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.602785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.603056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.603843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] RBAC: Adding query filter params (role.id=a449c88b1754437d942b32f2ec39cb9b, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=41d9317e24714de1a310198bb35eb00b) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:59.604149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:59.605602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:59.611747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 527/1048] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:08:59 2022] GET /identity/v3/role_assignments?user.id=41d9317e24714de1a310198bb35eb00b&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=a449c88b1754437d942b32f2ec39cb9b => generated 260 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:59.617905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.618541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.618819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.619090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.619819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:08:59.620121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:59.621555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:59.637783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 522/1049] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:08:59 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:08:59.639520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:59.759134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287290e-50cc-495a-b17d-af8bfe1739fd None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.782676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:59.797389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3965012-ea1d-4c7e-b0df-e019fb684bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.798725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965012-ea1d-4c7e-b0df-e019fb684bb0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.799007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965012-ea1d-4c7e-b0df-e019fb684bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.799394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965012-ea1d-4c7e-b0df-e019fb684bb0 None admin] PATH_INFO: `/v3/users/3d2ec850425447659bf2cd43bacbec2c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:08:59.813464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3965012-ea1d-4c7e-b0df-e019fb684bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=3d2ec850425447659bf2cd43bacbec2c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:08:59.815248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3965012-ea1d-4c7e-b0df-e019fb684bb0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:08:59.821887 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 528/1050] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:08:59 2022] GET /identity/v3/users/3d2ec850425447659bf2cd43bacbec2c => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:08:59.823240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:08:59.925279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6061-6f78-4219-9e17-312d9b068b4b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:08:59.952411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:08:59.961865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068bbad-42b3-444a-b7f4-17436587d36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:08:59.962482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068bbad-42b3-444a-b7f4-17436587d36c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:08:59.962822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068bbad-42b3-444a-b7f4-17436587d36c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:08:59.963095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068bbad-42b3-444a-b7f4-17436587d36c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:00.029307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068bbad-42b3-444a-b7f4-17436587d36c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:00.030701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068bbad-42b3-444a-b7f4-17436587d36c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:00.146990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 523/1051] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:08:59 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:00.148310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3965012-ea1d-4c7e-b0df-e019fb684bb0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:00.252939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3965012-ea1d-4c7e-b0df-e019fb684bb0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:00.288328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d93ac2-2f7f-4f19-8127-e428ea8df6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:00.288895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d93ac2-2f7f-4f19-8127-e428ea8df6d6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:00.289170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d93ac2-2f7f-4f19-8127-e428ea8df6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:00.289462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d93ac2-2f7f-4f19-8127-e428ea8df6d6 None admin] PATH_INFO: `/v3/projects/3a2c85ae58814c1c99589e2865f3adbd` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:00.290627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d93ac2-2f7f-4f19-8127-e428ea8df6d6 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2c85ae58814c1c99589e2865f3adbd)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:00.292055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d93ac2-2f7f-4f19-8127-e428ea8df6d6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:00.299372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 529/1052] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:00 2022] GET /identity/v3/projects/3a2c85ae58814c1c99589e2865f3adbd => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:00.300661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068bbad-42b3-444a-b7f4-17436587d36c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:00.330711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:00.405089 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068bbad-42b3-444a-b7f4-17436587d36c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:00.441684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6fccc-adbd-4a0b-ae7e-d448bf167bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:00.442669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fccc-adbd-4a0b-ae7e-d448bf167bb6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:00.443119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fccc-adbd-4a0b-ae7e-d448bf167bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:00.443565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fccc-adbd-4a0b-ae7e-d448bf167bb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:00.444852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6fccc-adbd-4a0b-ae7e-d448bf167bb6 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=3a2c85ae58814c1c99589e2865f3adbd, user.id=3d2ec850425447659bf2cd43bacbec2c) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:00.445328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6fccc-adbd-4a0b-ae7e-d448bf167bb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:00.447638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6fccc-adbd-4a0b-ae7e-d448bf167bb6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:00.461685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 524/1053] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:00 2022] GET /identity/v3/role_assignments?user.id=3d2ec850425447659bf2cd43bacbec2c&scope.project.id=3a2c85ae58814c1c99589e2865f3adbd&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 260 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:01.229121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952e2c7-d311-4068-9552-651f38275de8 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.229687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952e2c7-d311-4068-9552-651f38275de8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.229687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952e2c7-d311-4068-9552-651f38275de8 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.231061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 530/1054] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:01.236389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b0fee-eeb5-4ac6-9b79-ff72e0d29ac9 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.236815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b0fee-eeb5-4ac6-9b79-ff72e0d29ac9 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.236987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b0fee-eeb5-4ac6-9b79-ff72e0d29ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.255954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2a4b0fee-eeb5-4ac6-9b79-ff72e0d29ac9 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:01.257127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2a4b0fee-eeb5-4ac6-9b79-ff72e0d29ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:01.324034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 525/1055] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:01 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:01.346916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952e2c7-d311-4068-9552-651f38275de8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:01.395335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:01.449391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952e2c7-d311-4068-9552-651f38275de8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:01.482774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68942a76-e5d5-47e6-a5a3-64b88fe4280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:01.483409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942a76-e5d5-47e6-a5a3-64b88fe4280b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.483682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942a76-e5d5-47e6-a5a3-64b88fe4280b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.483948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942a76-e5d5-47e6-a5a3-64b88fe4280b None admin] PATH_INFO: `/v3/services/image` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.484557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68942a76-e5d5-47e6-a5a3-64b88fe4280b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:01.485988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68942a76-e5d5-47e6-a5a3-64b88fe4280b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-68942a76-e5d5-47e6-a5a3-64b88fe4280b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:01.494784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:01.495769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 22:09:01.496627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:01.504416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 531/1056] 10.222.0.109 () {56 vars in 1110 bytes} [Thu Aug 4 22:09:01 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:01.504551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b0fee-eeb5-4ac6-9b79-ff72e0d29ac9 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:01.527015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:01.605074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b0fee-eeb5-4ac6-9b79-ff72e0d29ac9 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:01.632035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee755c3-f705-45f8-9331-a895ec021a14 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.632423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee755c3-f705-45f8-9331-a895ec021a14 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.632760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee755c3-f705-45f8-9331-a895ec021a14 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.634355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 532/1057] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:09:01.639753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2069037-fbea-4a73-9fbc-2c0f363d6027 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.640193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2069037-fbea-4a73-9fbc-2c0f363d6027 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.640614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2069037-fbea-4a73-9fbc-2c0f363d6027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.643388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:01.644002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.644279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.644544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.645210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] RBAC: Adding query filter params (name=image) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:01.645592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:01.646891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:01.651917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 526/1058] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:01 2022] GET /identity/v3/services?name=image => generated 123 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:01.657522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:01.662334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-c2069037-fbea-4a73-9fbc-2c0f363d6027 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:01.663505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c2069037-fbea-4a73-9fbc-2c0f363d6027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:01.729331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 533/1059] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:01 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:09:01.746435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:01.752546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2069037-fbea-4a73-9fbc-2c0f363d6027 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:01.770196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f681ed3-ef54-4313-88fe-18d51ea3b3c2 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:01.796489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:01.811371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:01.812079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.812364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.812598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.813959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] RBAC: Adding query filter params (type=image) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:01.814282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:01.815809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:01.821102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 527/1060] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:01 2022] GET /identity/v3/services?type=image => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:01.826388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:01.864181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2069037-fbea-4a73-9fbc-2c0f363d6027 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:01.898527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d7d08-c272-4d81-b2e6-566f1bf20154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:01.899086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d7d08-c272-4d81-b2e6-566f1bf20154 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.899363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d7d08-c272-4d81-b2e6-566f1bf20154 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.899632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d7d08-c272-4d81-b2e6-566f1bf20154 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.909445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d7d08-c272-4d81-b2e6-566f1bf20154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-b51d7d08-c272-4d81-b2e6-566f1bf20154 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:01.914283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:01.915397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:09:01.916443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:01.920274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 534/1061] 10.222.0.109 () {56 vars in 1125 bytes} [Thu Aug 4 22:09:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:09:01.921156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d7d08-c272-4d81-b2e6-566f1bf20154 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:01.935306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e630454-f000-4165-843b-3bef46a0a6d9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:01.973642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d9955-1c17-457e-85b2-224b547d76f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:01.974524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d9955-1c17-457e-85b2-224b547d76f5 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:01.975124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d9955-1c17-457e-85b2-224b547d76f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:01.975402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d9955-1c17-457e-85b2-224b547d76f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:01.976170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d9955-1c17-457e-85b2-224b547d76f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:01.977690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d9955-1c17-457e-85b2-224b547d76f5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:02.009448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 528/1062] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:09:01 2022] POST /identity/v3/endpoints => generated 329 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:09:02.038701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:02.041895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d7d08-c272-4d81-b2e6-566f1bf20154 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:02.099212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:02.099951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.100235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.100505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.101243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:02.101608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:02.103014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:02.116851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 535/1063] 10.222.0.109 () {56 vars in 1120 bytes} [Thu Aug 4 22:09:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:02.122620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d9955-1c17-457e-85b2-224b547d76f5 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:02.133395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:02.237330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d9955-1c17-457e-85b2-224b547d76f5 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:02.275696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:02.276264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.276543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.276821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.288193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:02.289671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:09:02.304996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:02.305962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:02.306862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:09:02.306862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:09:02.306862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:02.306862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:02.306862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 529/1064] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:02.312982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:02.415480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b51e-d7a8-4796-886b-66adf734a582 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:02.449489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acc91a-f24a-4732-8334-2aadd8834066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:02.450116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc91a-f24a-4732-8334-2aadd8834066 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.450401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc91a-f24a-4732-8334-2aadd8834066 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.450672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc91a-f24a-4732-8334-2aadd8834066 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.451356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acc91a-f24a-4732-8334-2aadd8834066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:02.451656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acc91a-f24a-4732-8334-2aadd8834066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:02.453010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acc91a-f24a-4732-8334-2aadd8834066 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:02.469685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 536/1065] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:09:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:02.471145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:02.530753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829c64-73ac-4dac-ad57-e2a663ddb550 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.531053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829c64-73ac-4dac-ad57-e2a663ddb550 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.531235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829c64-73ac-4dac-ad57-e2a663ddb550 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.532789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 537/1066] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:09:02.535693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:02.538183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592df0-b9bb-4c0a-82ad-066e26d21768 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.538474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592df0-b9bb-4c0a-82ad-066e26d21768 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.538750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592df0-b9bb-4c0a-82ad-066e26d21768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.557702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2d592df0-b9bb-4c0a-82ad-066e26d21768 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:02.558994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2d592df0-b9bb-4c0a-82ad-066e26d21768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:02.573723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:02.599812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49155e54-24f3-402f-9d57-7c4416b165f1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:02.633451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 538/1067] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:02 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:09:02.643930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:02.644644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.644996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.645334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.653079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:02.654656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:02.654951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d592df0-b9bb-4c0a-82ad-066e26d21768 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:09:02.667538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:02.668508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:02.670061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 530/1068] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:09:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:09:02.674938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:02.763064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d592df0-b9bb-4c0a-82ad-066e26d21768 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:02.777109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba5741-b14d-4ac3-898e-60f02a60c98a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:02.797145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:02.803668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbebedd-8572-4914-be03-c66dd2130230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:02.804790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbebedd-8572-4914-be03-c66dd2130230 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.805250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbebedd-8572-4914-be03-c66dd2130230 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.805724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbebedd-8572-4914-be03-c66dd2130230 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.817565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:02.818343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.818522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbebedd-8572-4914-be03-c66dd2130230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:02.818623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.818891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.819692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:02.820047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:02.821646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-6dbebedd-8572-4914-be03-c66dd2130230 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:02.823357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:02.824273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:02.825128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 539/1069] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:09:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:09:02.831436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbebedd-8572-4914-be03-c66dd2130230 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:02.846750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 531/1070] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:09:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:02.846920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:02.847743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:02.953434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23998c-b6e1-4f6a-bcce-fc8395dd32a4 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:02.958290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbebedd-8572-4914-be03-c66dd2130230 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:02.991008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:02.991653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.991931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.992201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.992535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:02.993142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:02.993434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:02.993727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:02.994430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:02.994731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:02.996086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.003619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:03.005193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 540/1071] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:09:02 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:03.005311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.010724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:09:03.026377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:03.027336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:03.028238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:09:03.028238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:09:03.028238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:03.028238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:03.028238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 532/1072] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:09:03.033199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:03.067881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:03.088331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:03.124985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e31e-c66d-449f-b88e-ad71c43f0c9b None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:03.154042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964b8d2-71db-4fa2-b9d9-076016ec84b9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:03.162247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410318d-1083-4c7b-8cff-2402441f9c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:03.163012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410318d-1083-4c7b-8cff-2402441f9c7d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:03.163289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410318d-1083-4c7b-8cff-2402441f9c7d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:03.163557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410318d-1083-4c7b-8cff-2402441f9c7d None admin] PATH_INFO: `/v3/users/3d2ec850425447659bf2cd43bacbec2c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:03.164943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410318d-1083-4c7b-8cff-2402441f9c7d None admin] RBAC: Authorizing `identity:get_user(user_id=3d2ec850425447659bf2cd43bacbec2c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:03.166495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410318d-1083-4c7b-8cff-2402441f9c7d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.173397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 541/1073] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:03 2022] GET /identity/v3/users/3d2ec850425447659bf2cd43bacbec2c => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:03.174518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410318d-1083-4c7b-8cff-2402441f9c7d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:03.197485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:03.198114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:03.198393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:03.198670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:03.199363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:03.199667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:03.201049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.215035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 533/1074] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:09:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:03.221022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:03.284900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410318d-1083-4c7b-8cff-2402441f9c7d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:03.320023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:03.325501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b343e2b-34b5-476d-aafd-bcc22e6dd9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:03.326579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b343e2b-34b5-476d-aafd-bcc22e6dd9f0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:03.327015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b343e2b-34b5-476d-aafd-bcc22e6dd9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:03.327283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b343e2b-34b5-476d-aafd-bcc22e6dd9f0 None admin] PATH_INFO: `/v3/projects/3a2c85ae58814c1c99589e2865f3adbd` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:03.328442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b343e2b-34b5-476d-aafd-bcc22e6dd9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2c85ae58814c1c99589e2865f3adbd)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:03.330172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b343e2b-34b5-476d-aafd-bcc22e6dd9f0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.332191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 542/1075] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:03 2022] GET /identity/v3/projects/3a2c85ae58814c1c99589e2865f3adbd => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:03.333301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089993c-e312-4248-8072-838d14e6b34c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:03.339069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b343e2b-34b5-476d-aafd-bcc22e6dd9f0 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:03.349307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:03.374647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:03.375221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:03.375503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:03.376521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:03.384041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:03.385535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:09:03.397275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:03.398316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 534/1076] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:03.404048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:03.450121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b343e2b-34b5-476d-aafd-bcc22e6dd9f0 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:03.488934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f9c9c-a75d-459b-b33a-93d02b732930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:03.489635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f9c9c-a75d-459b-b33a-93d02b732930 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:03.489919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f9c9c-a75d-459b-b33a-93d02b732930 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:03.490197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f9c9c-a75d-459b-b33a-93d02b732930 None admin] PATH_INFO: `/v3/projects/3a2c85ae58814c1c99589e2865f3adbd/users/3d2ec850425447659bf2cd43bacbec2c/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:03.492180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f9c9c-a75d-459b-b33a-93d02b732930 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a2c85ae58814c1c99589e2865f3adbd, user_id=3d2ec850425447659bf2cd43bacbec2c, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:03.493654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f9c9c-a75d-459b-b33a-93d02b732930 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.507396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 543/1077] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:09:03 2022] PUT /identity/v3/projects/3a2c85ae58814c1c99589e2865f3adbd/users/3d2ec850425447659bf2cd43bacbec2c/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:09:03.508266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a9a4a-8d22-40ca-9a2f-350820abff74 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:03.556216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b68f0-37ea-4f9e-b6dc-1453c994986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:03.556858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b68f0-37ea-4f9e-b6dc-1453c994986c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:03.557166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b68f0-37ea-4f9e-b6dc-1453c994986c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:03.557455 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b68f0-37ea-4f9e-b6dc-1453c994986c None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:03.558232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b68f0-37ea-4f9e-b6dc-1453c994986c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:03.558538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b68f0-37ea-4f9e-b6dc-1453c994986c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:03.560059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b68f0-37ea-4f9e-b6dc-1453c994986c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.573544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 535/1078] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:03.579619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f9c9c-a75d-459b-b33a-93d02b732930 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:03.658817 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:03.699632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f9c9c-a75d-459b-b33a-93d02b732930 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:03.734597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb1770-22d8-44e6-bd57-7184e9e9ce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:03.735992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb1770-22d8-44e6-bd57-7184e9e9ce3c None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:03.736456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb1770-22d8-44e6-bd57-7184e9e9ce3c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:03.736539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb1770-22d8-44e6-bd57-7184e9e9ce3c None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/41d9317e24714de1a310198bb35eb00b/roles/a449c88b1754437d942b32f2ec39cb9b` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:03.738709 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb1770-22d8-44e6-bd57-7184e9e9ce3c None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=41d9317e24714de1a310198bb35eb00b, role_id=a449c88b1754437d942b32f2ec39cb9b)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:03.740163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb1770-22d8-44e6-bd57-7184e9e9ce3c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:03.755783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 544/1079] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:09:03 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/41d9317e24714de1a310198bb35eb00b/roles/a449c88b1754437d942b32f2ec39cb9b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:09:04.081128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177658-7ede-4b3c-8250-b7c7f7e315c5 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:04.081535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177658-7ede-4b3c-8250-b7c7f7e315c5 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:04.081734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177658-7ede-4b3c-8250-b7c7f7e315c5 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:04.083108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 536/1080] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:04 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:04.088423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d3be0-201f-4e2f-a4f5-415ce14abc29 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:04.088785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d3be0-201f-4e2f-a4f5-415ce14abc29 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:04.089136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d3be0-201f-4e2f-a4f5-415ce14abc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:04.108455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2d7d3be0-201f-4e2f-a4f5-415ce14abc29 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:04.109699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2d7d3be0-201f-4e2f-a4f5-415ce14abc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:04.181458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 545/1081] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:04 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:04.201975 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177658-7ede-4b3c-8250-b7c7f7e315c5 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:04.214722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:04.309781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177658-7ede-4b3c-8250-b7c7f7e315c5 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:04.344326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25a62e-8851-4b28-aefc-16ee5d531dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:04.345110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25a62e-8851-4b28-aefc-16ee5d531dec None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:04.345302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25a62e-8851-4b28-aefc-16ee5d531dec None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:04.345645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25a62e-8851-4b28-aefc-16ee5d531dec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:04.411964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25a62e-8851-4b28-aefc-16ee5d531dec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:04.413504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25a62e-8851-4b28-aefc-16ee5d531dec None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:04.422967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:04.518032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 537/1082] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:04 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:04.519769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d3be0-201f-4e2f-a4f5-415ce14abc29 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:04.576804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:04.623431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d3be0-201f-4e2f-a4f5-415ce14abc29 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:04.660170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e36d16-1434-494c-adf5-20738da724fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:04.660819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e36d16-1434-494c-adf5-20738da724fb None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:04.661100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e36d16-1434-494c-adf5-20738da724fb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:04.661373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e36d16-1434-494c-adf5-20738da724fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:04.671611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e36d16-1434-494c-adf5-20738da724fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:04.673610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e36d16-1434-494c-adf5-20738da724fb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e4e36d16-1434-494c-adf5-20738da724fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:09:04.689659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:04.690572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:04.691459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:09:04.691459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:09:04.691459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:04.691459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:04.693755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 546/1083] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:09:04.696709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25a62e-8851-4b28-aefc-16ee5d531dec None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:04.802392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25a62e-8851-4b28-aefc-16ee5d531dec None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:04.815861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:04.842968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368e749-41b6-4b7a-882a-2736cbd84ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:04.843589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e749-41b6-4b7a-882a-2736cbd84ca3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:04.843869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e749-41b6-4b7a-882a-2736cbd84ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:04.844143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e749-41b6-4b7a-882a-2736cbd84ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:04.845367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e749-41b6-4b7a-882a-2736cbd84ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:04.845705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e749-41b6-4b7a-882a-2736cbd84ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:04.847074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e749-41b6-4b7a-882a-2736cbd84ca3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:04.864170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 538/1084] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:09:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:05.561133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84469d5-7d37-4f5d-9c4a-0a4b7ad2fa4b None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:05.561719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84469d5-7d37-4f5d-9c4a-0a4b7ad2fa4b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:05.561719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84469d5-7d37-4f5d-9c4a-0a4b7ad2fa4b None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:05.563664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 547/1085] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:05.569130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583d0a-1ab1-47f7-83d5-f3e2893a2b57 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:05.569578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583d0a-1ab1-47f7-83d5-f3e2893a2b57 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:05.569700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583d0a-1ab1-47f7-83d5-f3e2893a2b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:05.588326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-51583d0a-1ab1-47f7-83d5-f3e2893a2b57 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:05.589537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-51583d0a-1ab1-47f7-83d5-f3e2893a2b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:05.654898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 539/1086] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:05 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:09:05.678064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84469d5-7d37-4f5d-9c4a-0a4b7ad2fa4b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:05.757906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4aec-8d8e-4d52-a53e-6b69783ffa8f None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:05.758192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4aec-8d8e-4d52-a53e-6b69783ffa8f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:05.758460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4aec-8d8e-4d52-a53e-6b69783ffa8f None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:05.759808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 540/1087] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:09:05.764389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba9b5-12a7-4805-972d-176736d95315 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:05.764664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba9b5-12a7-4805-972d-176736d95315 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:05.764929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba9b5-12a7-4805-972d-176736d95315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:05.778913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84469d5-7d37-4f5d-9c4a-0a4b7ad2fa4b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:05.783379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-c46ba9b5-12a7-4805-972d-176736d95315 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:05.784633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c46ba9b5-12a7-4805-972d-176736d95315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:05.788204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:05.814915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebe4f2-feee-4d20-8c92-5485ab908c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:05.815523 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe4f2-feee-4d20-8c92-5485ab908c84 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:05.815800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe4f2-feee-4d20-8c92-5485ab908c84 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:05.816078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe4f2-feee-4d20-8c92-5485ab908c84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:05.827174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ebe4f2-feee-4d20-8c92-5485ab908c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-68ebe4f2-feee-4d20-8c92-5485ab908c84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:05.832616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:05.833651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:05.834505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 548/1088] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:09:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:05.840249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebe4f2-feee-4d20-8c92-5485ab908c84 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:05.855940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 541/1089] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:05 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:09:05.878863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ba9b5-12a7-4805-972d-176736d95315 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:05.905717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:05.942551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebe4f2-feee-4d20-8c92-5485ab908c84 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:05.976517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:05.976879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ba9b5-12a7-4805-972d-176736d95315 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:05.977149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:05.977450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:05.977732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:05.978441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:05.978740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:05.980110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:05.989864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 549/1090] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:09:05 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:05.996124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.012966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224fdd4-4f97-4a41-9093-ca4842c0e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.013642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224fdd4-4f97-4a41-9093-ca4842c0e91b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.013920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224fdd4-4f97-4a41-9093-ca4842c0e91b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.014213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224fdd4-4f97-4a41-9093-ca4842c0e91b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.024828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224fdd4-4f97-4a41-9093-ca4842c0e91b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:06.026165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-f224fdd4-4f97-4a41-9093-ca4842c0e91b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:06.029706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:06.030824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:06.032201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 542/1091] 10.222.0.109 () {56 vars in 1125 bytes} [Thu Aug 4 22:09:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:09:06.039816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224fdd4-4f97-4a41-9093-ca4842c0e91b None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.106844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b582a-0c8a-4ef6-ac1d-17a17dee00ef None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.134213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:06.142036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ea65d-ca5e-4801-bc81-251da051958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.142693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ea65d-ca5e-4801-bc81-251da051958d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.142971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ea65d-ca5e-4801-bc81-251da051958d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.143240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ea65d-ca5e-4801-bc81-251da051958d None admin] PATH_INFO: `/v3/users/3d2ec850425447659bf2cd43bacbec2c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.144432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ea65d-ca5e-4801-bc81-251da051958d None admin] RBAC: Authorizing `identity:get_user(user_id=3d2ec850425447659bf2cd43bacbec2c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.145913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ea65d-ca5e-4801-bc81-251da051958d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.147520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 550/1092] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:05 2022] GET /identity/v3/users/3d2ec850425447659bf2cd43bacbec2c => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:06.148000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224fdd4-4f97-4a41-9093-ca4842c0e91b None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.153126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ea65d-ca5e-4801-bc81-251da051958d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.184928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.185573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.185846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.186113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.186810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:06.187109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.188807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.196431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 543/1093] 10.222.0.109 () {56 vars in 1120 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:06.201952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.256127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:06.259743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ea65d-ca5e-4801-bc81-251da051958d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.295318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbce69-2fcc-4411-9371-b55af68ce163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.295928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbce69-2fcc-4411-9371-b55af68ce163 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.296205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbce69-2fcc-4411-9371-b55af68ce163 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.296474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbce69-2fcc-4411-9371-b55af68ce163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.304566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b0cd1-2689-479e-8e2d-a1f54e72fe2a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.338539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.339265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.339573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.339845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.350980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.352432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.362708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbce69-2fcc-4411-9371-b55af68ce163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.364076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbce69-2fcc-4411-9371-b55af68ce163 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:09:06.366373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:06.367299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:06.368175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:09:06.368175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:09:06.368175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:06.368175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:06.368175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 544/1094] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:09:06.373136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.375901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:06.464008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:06.468707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 551/1095] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 22:09:06.474515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbce69-2fcc-4411-9371-b55af68ce163 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.475756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcce76-6af7-44b2-9f8c-8d3d9b8150e0 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.510057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.510675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.510958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.511235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.511916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:06.512216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.513611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.526054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 545/1096] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:06.531556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.571793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbce69-2fcc-4411-9371-b55af68ce163 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.596016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:06.606176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f57da-e8c7-4a91-8cfa-1ee3573f65fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.606808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f57da-e8c7-4a91-8cfa-1ee3573f65fb None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.607081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f57da-e8c7-4a91-8cfa-1ee3573f65fb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.607407 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f57da-e8c7-4a91-8cfa-1ee3573f65fb None admin] PATH_INFO: `/v3/projects/3a2c85ae58814c1c99589e2865f3adbd` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.608572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f57da-e8c7-4a91-8cfa-1ee3573f65fb None admin] RBAC: Authorizing `identity:get_project(project_id=3a2c85ae58814c1c99589e2865f3adbd)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.610081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f57da-e8c7-4a91-8cfa-1ee3573f65fb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.611824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 552/1097] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/projects/3a2c85ae58814c1c99589e2865f3adbd => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:06.617796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f57da-e8c7-4a91-8cfa-1ee3573f65fb None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.633673 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ab11-c455-45bb-9384-fdc24ce462e5 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.667914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.668548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.668821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.669086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.676234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.677699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.681469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:09:06.689882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:06.690849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:06.691788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 546/1098] 10.222.0.109 () {56 vars in 1098 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:09:06.696638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.717535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f57da-e8c7-4a91-8cfa-1ee3573f65fb None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.750123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.751483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.751739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.752006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.752792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=3a2c85ae58814c1c99589e2865f3adbd, user.id=3d2ec850425447659bf2cd43bacbec2c) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:06.753093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.754561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.760278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 553/1099] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/role_assignments?user.id=3d2ec850425447659bf2cd43bacbec2c&scope.project.id=3a2c85ae58814c1c99589e2865f3adbd&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 610 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:06.798554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fb5a8-3181-4881-8965-d25ac51540e4 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:06.823542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:06.838019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:06.838653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:06.838937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:06.839210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:06.840021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:06.840329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:06.841864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:06.861162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 547/1100] 10.222.0.109 () {56 vars in 1138 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:06.867252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:06.972681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1544-7311-446a-ae99-006bbec1a61e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:07.002512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:07.007592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66933be-0591-41f7-a412-809067e4128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:07.008626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66933be-0591-41f7-a412-809067e4128e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:07.008903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66933be-0591-41f7-a412-809067e4128e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:07.009203 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66933be-0591-41f7-a412-809067e4128e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:07.075158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66933be-0591-41f7-a412-809067e4128e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:07.076615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66933be-0591-41f7-a412-809067e4128e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:07.175936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 554/1101] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:06 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:07.182125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:07.281976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c071c1-9be9-4660-99b2-7c0ccaae7ace None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:07.315820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:07.316450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:07.316721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:07.316986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:07.326689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:07.328126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:09:07.342682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:07.343697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:07.344685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:09:07.344685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:09:07.344685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:09:07.344685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:07.344685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 548/1102] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:07.349500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66933be-0591-41f7-a412-809067e4128e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:07.381483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:07.449501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66933be-0591-41f7-a412-809067e4128e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:07.482991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:07.483604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:07.483878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:07.484143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:07.485014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:07.485306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:07.486687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:07.497816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 555/1103] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:09:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:07.504443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:07.537032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:07.603580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ecd7e-3be0-40ef-9a09-964784fc1070 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:07.636571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:07.637191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:07.637494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:07.637765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:07.644628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:07.646086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:09:07.657092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:07.658108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:07.659013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 549/1104] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:07.663871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:07.752470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:07.762357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56978ceb-2c08-4557-8c65-dfc7bdd52f09 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:07.794642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bc418-a690-4adb-91a1-e3dc82f601cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:07.795183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bc418-a690-4adb-91a1-e3dc82f601cf None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:07.795491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bc418-a690-4adb-91a1-e3dc82f601cf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:07.795759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bc418-a690-4adb-91a1-e3dc82f601cf None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:07.796494 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bc418-a690-4adb-91a1-e3dc82f601cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:07.796799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bc418-a690-4adb-91a1-e3dc82f601cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:07.798797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bc418-a690-4adb-91a1-e3dc82f601cf None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:07.810490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 556/1105] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:07.817213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:07.914222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:07.924250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392754e8-c0ba-45c6-9cf4-d465479608e6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:07.957882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66121-d23c-4f5d-be30-82f9e7c189e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:07.958491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66121-d23c-4f5d-be30-82f9e7c189e2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:07.958767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66121-d23c-4f5d-be30-82f9e7c189e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:07.959037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66121-d23c-4f5d-be30-82f9e7c189e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:07.959815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66121-d23c-4f5d-be30-82f9e7c189e2 None admin] RBAC: Adding query filter params (role.id=a449c88b1754437d942b32f2ec39cb9b, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=41d9317e24714de1a310198bb35eb00b) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:07.960121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66121-d23c-4f5d-be30-82f9e7c189e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:07.961583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66121-d23c-4f5d-be30-82f9e7c189e2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:07.967300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 550/1106] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:07 2022] GET /identity/v3/role_assignments?user.id=41d9317e24714de1a310198bb35eb00b&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=a449c88b1754437d942b32f2ec39cb9b => generated 610 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:08.798955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e7513-3ebe-438a-80ca-8e2fda93ab22 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:08.799477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e7513-3ebe-438a-80ca-8e2fda93ab22 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:08.799477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e7513-3ebe-438a-80ca-8e2fda93ab22 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:08.800846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 557/1107] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:08.806216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81798ef5-bf0e-4456-aa0c-c113c626ba10 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:08.806627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81798ef5-bf0e-4456-aa0c-c113c626ba10 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:08.806780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81798ef5-bf0e-4456-aa0c-c113c626ba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:08.825155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-81798ef5-bf0e-4456-aa0c-c113c626ba10 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:08.826377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-81798ef5-bf0e-4456-aa0c-c113c626ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:08.893826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 551/1108] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:08 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:08.918825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e7513-3ebe-438a-80ca-8e2fda93ab22 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:09.020837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e7513-3ebe-438a-80ca-8e2fda93ab22 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:09.053236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe7c7a-0435-49a2-967d-497c40a4fab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:09.053829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe7c7a-0435-49a2-967d-497c40a4fab5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:09.054113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe7c7a-0435-49a2-967d-497c40a4fab5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:09.054385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe7c7a-0435-49a2-967d-497c40a4fab5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:09.055530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe7c7a-0435-49a2-967d-497c40a4fab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:09.056947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe7c7a-0435-49a2-967d-497c40a4fab5 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:09.058544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 558/1109] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:08 2022] GET /identity/v3/domains/default => generated 200 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:09.065146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81798ef5-bf0e-4456-aa0c-c113c626ba10 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:09.150061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:09.165209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81798ef5-bf0e-4456-aa0c-c113c626ba10 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:09.198399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8686b7a-a716-4e59-87fb-629355feb67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:09.199004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8686b7a-a716-4e59-87fb-629355feb67a None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:09.199278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8686b7a-a716-4e59-87fb-629355feb67a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:09.199546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8686b7a-a716-4e59-87fb-629355feb67a None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:09.200236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8686b7a-a716-4e59-87fb-629355feb67a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:09.201685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8686b7a-a716-4e59-87fb-629355feb67a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:09.221141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 552/1110] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:09:09 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:09:11.224370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4656-2d63-415c-a40d-8d8ce0acd52f None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:11.224905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4656-2d63-415c-a40d-8d8ce0acd52f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:11.224905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4656-2d63-415c-a40d-8d8ce0acd52f None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:11.226310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 559/1111] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:11.231264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73df6da-ef71-4873-a8ea-adfc582e2080 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:11.231667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73df6da-ef71-4873-a8ea-adfc582e2080 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:11.231819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73df6da-ef71-4873-a8ea-adfc582e2080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:11.250556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b73df6da-ef71-4873-a8ea-adfc582e2080 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:11.251871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b73df6da-ef71-4873-a8ea-adfc582e2080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:11.317748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 553/1112] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:11 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:11.341849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba4656-2d63-415c-a40d-8d8ce0acd52f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:11.362012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:11.443723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba4656-2d63-415c-a40d-8d8ce0acd52f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:11.477594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cac06-4296-41ee-8696-53d0941eafd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:11.478212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cac06-4296-41ee-8696-53d0941eafd4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:11.478485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cac06-4296-41ee-8696-53d0941eafd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:11.478754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cac06-4296-41ee-8696-53d0941eafd4 None admin] PATH_INFO: `/v3/roles/8f997477100248a9ace1cc28831caaac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:11.480188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cac06-4296-41ee-8696-53d0941eafd4 None admin] RBAC: Authorizing `identity:get_role(role_id=8f997477100248a9ace1cc28831caaac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:11.481575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cac06-4296-41ee-8696-53d0941eafd4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:11.482896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 560/1113] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:11 2022] GET /identity/v3/roles/8f997477100248a9ace1cc28831caaac => generated 224 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:11.488835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73df6da-ef71-4873-a8ea-adfc582e2080 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:11.555805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:11.585526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73df6da-ef71-4873-a8ea-adfc582e2080 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:11.618630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a0c6-2f37-4869-a473-f2d13c9ea0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:11.619195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a0c6-2f37-4869-a473-f2d13c9ea0d1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:11.619467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a0c6-2f37-4869-a473-f2d13c9ea0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:11.619756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a0c6-2f37-4869-a473-f2d13c9ea0d1 None admin] PATH_INFO: `/v3/users/c2a81aadcb0b420a9b41d12d63bd3e25` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:11.632223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5a0c6-2f37-4869-a473-f2d13c9ea0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a81aadcb0b420a9b41d12d63bd3e25)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:11.633689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5a0c6-2f37-4869-a473-f2d13c9ea0d1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:11.635285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 554/1114] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:11 2022] GET /identity/v3/users/c2a81aadcb0b420a9b41d12d63bd3e25 => generated 287 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:11.641761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cac06-4296-41ee-8696-53d0941eafd4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:11.722146 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:11.736537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cac06-4296-41ee-8696-53d0941eafd4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:11.769586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615677ad-fc0b-4d29-9c4d-077e9c9da1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:11.770180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615677ad-fc0b-4d29-9c4d-077e9c9da1c1 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:11.770456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615677ad-fc0b-4d29-9c4d-077e9c9da1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:11.770723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615677ad-fc0b-4d29-9c4d-077e9c9da1c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:11.835298 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615677ad-fc0b-4d29-9c4d-077e9c9da1c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:11.836730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615677ad-fc0b-4d29-9c4d-077e9c9da1c1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:11.917286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:11.932196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 561/1115] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:11 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:11.938742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a0c6-2f37-4869-a473-f2d13c9ea0d1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:12.035376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a0c6-2f37-4869-a473-f2d13c9ea0d1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:12.062226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:12.068894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7cc66-3996-4e05-a0b8-5228a68bc705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:12.069554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7cc66-3996-4e05-a0b8-5228a68bc705 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:12.069832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7cc66-3996-4e05-a0b8-5228a68bc705 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:12.070098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7cc66-3996-4e05-a0b8-5228a68bc705 None admin] PATH_INFO: `/v3/projects/3a2c85ae58814c1c99589e2865f3adbd` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:12.071223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7cc66-3996-4e05-a0b8-5228a68bc705 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2c85ae58814c1c99589e2865f3adbd)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:12.072644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7cc66-3996-4e05-a0b8-5228a68bc705 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:12.074344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 555/1116] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:11 2022] GET /identity/v3/projects/3a2c85ae58814c1c99589e2865f3adbd => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:12.081152 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615677ad-fc0b-4d29-9c4d-077e9c9da1c1 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:12.177277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615677ad-fc0b-4d29-9c4d-077e9c9da1c1 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:12.210405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241cf6e-ca5a-4343-b97e-3e12214ebc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:12.211015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241cf6e-ca5a-4343-b97e-3e12214ebc67 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:12.211295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241cf6e-ca5a-4343-b97e-3e12214ebc67 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:12.211571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241cf6e-ca5a-4343-b97e-3e12214ebc67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:12.212344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241cf6e-ca5a-4343-b97e-3e12214ebc67 None admin] RBAC: Adding query filter params (role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=3a2c85ae58814c1c99589e2865f3adbd, user.id=c2a81aadcb0b420a9b41d12d63bd3e25) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:12.212650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241cf6e-ca5a-4343-b97e-3e12214ebc67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:12.214108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241cf6e-ca5a-4343-b97e-3e12214ebc67 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:12.219494 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 562/1117] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:12 2022] GET /identity/v3/role_assignments?user.id=c2a81aadcb0b420a9b41d12d63bd3e25&scope.project.id=3a2c85ae58814c1c99589e2865f3adbd&role.id=8f997477100248a9ace1cc28831caaac => generated 260 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:14.193031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33ab2-84d6-4299-a486-e73114aa2b9d None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:14.193579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33ab2-84d6-4299-a486-e73114aa2b9d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:14.193694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33ab2-84d6-4299-a486-e73114aa2b9d None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:14.194986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 556/1118] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:14.200139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e246f-440a-4b6b-a298-a71f5bed4809 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:14.200561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e246f-440a-4b6b-a298-a71f5bed4809 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:14.200687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e246f-440a-4b6b-a298-a71f5bed4809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:14.219832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-b08e246f-440a-4b6b-a298-a71f5bed4809 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:14.221013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b08e246f-440a-4b6b-a298-a71f5bed4809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:14.268167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:14.288952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 563/1119] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:14 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:09:14.310968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f33ab2-84d6-4299-a486-e73114aa2b9d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:14.410371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f33ab2-84d6-4299-a486-e73114aa2b9d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:14.443742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbfc18-6194-4395-8ee9-0f41ec4419ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:14.444376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbfc18-6194-4395-8ee9-0f41ec4419ff None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:14.444654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbfc18-6194-4395-8ee9-0f41ec4419ff None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:14.444924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbfc18-6194-4395-8ee9-0f41ec4419ff None admin] PATH_INFO: `/v3/roles/8f997477100248a9ace1cc28831caaac` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:14.446401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdbfc18-6194-4395-8ee9-0f41ec4419ff None admin] RBAC: Authorizing `identity:get_role(role_id=8f997477100248a9ace1cc28831caaac)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:14.447767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdbfc18-6194-4395-8ee9-0f41ec4419ff None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:14.449117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 557/1120] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:14 2022] GET /identity/v3/roles/8f997477100248a9ace1cc28831caaac => generated 224 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:14.455117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e246f-440a-4b6b-a298-a71f5bed4809 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:14.553137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e246f-440a-4b6b-a298-a71f5bed4809 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:14.586569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc801a1-ef85-4f43-9871-cdfc007edb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:14.587208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc801a1-ef85-4f43-9871-cdfc007edb6f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:14.587484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc801a1-ef85-4f43-9871-cdfc007edb6f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:14.587755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc801a1-ef85-4f43-9871-cdfc007edb6f None admin] PATH_INFO: `/v3/users/c2a81aadcb0b420a9b41d12d63bd3e25` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:14.588916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc801a1-ef85-4f43-9871-cdfc007edb6f None admin] RBAC: Authorizing `identity:get_user(user_id=c2a81aadcb0b420a9b41d12d63bd3e25)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:14.590383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc801a1-ef85-4f43-9871-cdfc007edb6f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:14.591988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 564/1121] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:14 2022] GET /identity/v3/users/c2a81aadcb0b420a9b41d12d63bd3e25 => generated 287 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:14.597925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbfc18-6194-4395-8ee9-0f41ec4419ff None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:14.683009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:14.716979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbfc18-6194-4395-8ee9-0f41ec4419ff None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:14.749920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ed132-0dcd-4b87-91a9-350b8fe5bda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:14.750642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ed132-0dcd-4b87-91a9-350b8fe5bda5 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:14.750920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ed132-0dcd-4b87-91a9-350b8fe5bda5 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:14.751191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ed132-0dcd-4b87-91a9-350b8fe5bda5 None admin] PATH_INFO: `/v3/projects/3a2c85ae58814c1c99589e2865f3adbd` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:14.752332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ed132-0dcd-4b87-91a9-350b8fe5bda5 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2c85ae58814c1c99589e2865f3adbd)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:14.753793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ed132-0dcd-4b87-91a9-350b8fe5bda5 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:14.755431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 558/1122] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:14 2022] GET /identity/v3/projects/3a2c85ae58814c1c99589e2865f3adbd => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:14.761482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc801a1-ef85-4f43-9871-cdfc007edb6f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:14.810712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:14.862164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc801a1-ef85-4f43-9871-cdfc007edb6f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:14.896800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4498c5-5109-417d-aa4c-c0d0d4baf139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:14.897479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4498c5-5109-417d-aa4c-c0d0d4baf139 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:14.897760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4498c5-5109-417d-aa4c-c0d0d4baf139 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:14.898030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4498c5-5109-417d-aa4c-c0d0d4baf139 None admin] PATH_INFO: `/v3/projects/3a2c85ae58814c1c99589e2865f3adbd/users/c2a81aadcb0b420a9b41d12d63bd3e25/roles/8f997477100248a9ace1cc28831caaac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:14.900172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4498c5-5109-417d-aa4c-c0d0d4baf139 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a2c85ae58814c1c99589e2865f3adbd, user_id=c2a81aadcb0b420a9b41d12d63bd3e25, role_id=8f997477100248a9ace1cc28831caaac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:14.901622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4498c5-5109-417d-aa4c-c0d0d4baf139 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:14.914788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 565/1123] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:09:14 2022] PUT /identity/v3/projects/3a2c85ae58814c1c99589e2865f3adbd/users/c2a81aadcb0b420a9b41d12d63bd3e25/roles/8f997477100248a9ace1cc28831caaac => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:09:16.875454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713db03-f441-41a6-8ae5-153c3eea6d77 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:16.876040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713db03-f441-41a6-8ae5-153c3eea6d77 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:16.876040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713db03-f441-41a6-8ae5-153c3eea6d77 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:16.877357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 559/1124] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:16.882622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327df4a-56bb-4195-816a-853e5f59dacb None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:16.883013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327df4a-56bb-4195-816a-853e5f59dacb None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:16.883168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327df4a-56bb-4195-816a-853e5f59dacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:16.901333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2327df4a-56bb-4195-816a-853e5f59dacb None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:16.902541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2327df4a-56bb-4195-816a-853e5f59dacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:16.970679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 566/1125] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:16 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:16.994356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713db03-f441-41a6-8ae5-153c3eea6d77 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:17.090399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713db03-f441-41a6-8ae5-153c3eea6d77 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:17.123573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2234127-7685-4068-9733-2db73ae7ef44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:17.124203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2234127-7685-4068-9733-2db73ae7ef44 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:17.124478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2234127-7685-4068-9733-2db73ae7ef44 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:17.124785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2234127-7685-4068-9733-2db73ae7ef44 None admin] PATH_INFO: `/v3/roles/8f997477100248a9ace1cc28831caaac` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:17.128344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2234127-7685-4068-9733-2db73ae7ef44 None admin] RBAC: Authorizing `identity:get_role(role_id=8f997477100248a9ace1cc28831caaac)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:17.129748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2234127-7685-4068-9733-2db73ae7ef44 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:17.131090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 560/1126] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:16 2022] GET /identity/v3/roles/8f997477100248a9ace1cc28831caaac => generated 224 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:17.136937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327df4a-56bb-4195-816a-853e5f59dacb None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:17.176473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:17.235677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327df4a-56bb-4195-816a-853e5f59dacb None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:17.269696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b927a6-8f8a-44de-8782-450e97090a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:17.270327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b927a6-8f8a-44de-8782-450e97090a10 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:17.270610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b927a6-8f8a-44de-8782-450e97090a10 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:17.270881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b927a6-8f8a-44de-8782-450e97090a10 None admin] PATH_INFO: `/v3/users/c2a81aadcb0b420a9b41d12d63bd3e25` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:17.272042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b927a6-8f8a-44de-8782-450e97090a10 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a81aadcb0b420a9b41d12d63bd3e25)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:17.273490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b927a6-8f8a-44de-8782-450e97090a10 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:17.275078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 567/1127] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:17 2022] GET /identity/v3/users/c2a81aadcb0b420a9b41d12d63bd3e25 => generated 287 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:09:17.280790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2234127-7685-4068-9733-2db73ae7ef44 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:17.283632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:17.380210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2234127-7685-4068-9733-2db73ae7ef44 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:17.413060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714c83-013f-4e91-8f87-e56556f59fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:17.413684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714c83-013f-4e91-8f87-e56556f59fad None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:17.413967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714c83-013f-4e91-8f87-e56556f59fad None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:17.414240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714c83-013f-4e91-8f87-e56556f59fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:17.479439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95714c83-013f-4e91-8f87-e56556f59fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:17.480873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95714c83-013f-4e91-8f87-e56556f59fad None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:17.482630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:17.576903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 561/1128] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:17 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:17.583037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b927a6-8f8a-44de-8782-450e97090a10 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:17.679894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b927a6-8f8a-44de-8782-450e97090a10 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:17.686802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:17.713388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ac13c-5a69-4e4d-80b0-1cb8f54e39e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:17.714045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac13c-5a69-4e4d-80b0-1cb8f54e39e0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:17.714326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac13c-5a69-4e4d-80b0-1cb8f54e39e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:17.714602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac13c-5a69-4e4d-80b0-1cb8f54e39e0 None admin] PATH_INFO: `/v3/projects/3a2c85ae58814c1c99589e2865f3adbd` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:17.715757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ac13c-5a69-4e4d-80b0-1cb8f54e39e0 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2c85ae58814c1c99589e2865f3adbd)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:17.717182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ac13c-5a69-4e4d-80b0-1cb8f54e39e0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:17.719509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 568/1129] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:17 2022] GET /identity/v3/projects/3a2c85ae58814c1c99589e2865f3adbd => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:17.725600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714c83-013f-4e91-8f87-e56556f59fad None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:17.819027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714c83-013f-4e91-8f87-e56556f59fad None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:17.820487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:17.852523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e31f9-15f5-4b91-9974-578f88fc736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:17.853119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e31f9-15f5-4b91-9974-578f88fc736f None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:17.853394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e31f9-15f5-4b91-9974-578f88fc736f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:17.853691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e31f9-15f5-4b91-9974-578f88fc736f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:17.854456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e31f9-15f5-4b91-9974-578f88fc736f None admin] RBAC: Adding query filter params (role.id=8f997477100248a9ace1cc28831caaac, scope.project.id=3a2c85ae58814c1c99589e2865f3adbd, user.id=c2a81aadcb0b420a9b41d12d63bd3e25) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:17.854757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e31f9-15f5-4b91-9974-578f88fc736f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:17.856171 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e31f9-15f5-4b91-9974-578f88fc736f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:17.862446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 562/1130] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:17 2022] GET /identity/v3/role_assignments?user.id=c2a81aadcb0b420a9b41d12d63bd3e25&scope.project.id=3a2c85ae58814c1c99589e2865f3adbd&role.id=8f997477100248a9ace1cc28831caaac => generated 610 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:19.857348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af62d-c1fa-4f07-b9bf-f15bf85d364e None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:19.857914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af62d-c1fa-4f07-b9bf-f15bf85d364e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:19.857914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af62d-c1fa-4f07-b9bf-f15bf85d364e None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:19.859807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 569/1131] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:19.865143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6c3d-6672-4d0a-a280-0c3c780ec615 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:19.865593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6c3d-6672-4d0a-a280-0c3c780ec615 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:19.865730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6c3d-6672-4d0a-a280-0c3c780ec615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:19.883800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9fda6c3d-6672-4d0a-a280-0c3c780ec615 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:19.884982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9fda6c3d-6672-4d0a-a280-0c3c780ec615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:19.948846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 563/1132] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:19 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:19.980258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af62d-c1fa-4f07-b9bf-f15bf85d364e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:20.081581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af62d-c1fa-4f07-b9bf-f15bf85d364e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:20.116529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87040870-fa4e-4465-8bba-e4340080c25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:20.117168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040870-fa4e-4465-8bba-e4340080c25c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:20.117470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040870-fa4e-4465-8bba-e4340080c25c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:20.117744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040870-fa4e-4465-8bba-e4340080c25c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:20.118874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87040870-fa4e-4465-8bba-e4340080c25c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:20.120303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87040870-fa4e-4465-8bba-e4340080c25c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:20.121911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 570/1133] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:19 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:20.128327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda6c3d-6672-4d0a-a280-0c3c780ec615 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:20.220086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:20.225807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda6c3d-6672-4d0a-a280-0c3c780ec615 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:20.258951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a6e59-7d4d-4d63-b79d-d5c6fba93da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:20.259564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6e59-7d4d-4d63-b79d-d5c6fba93da8 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:20.259845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6e59-7d4d-4d63-b79d-d5c6fba93da8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:20.260118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6e59-7d4d-4d63-b79d-d5c6fba93da8 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:20.260795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a6e59-7d4d-4d63-b79d-d5c6fba93da8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:20.262258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a6e59-7d4d-4d63-b79d-d5c6fba93da8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:20.288652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 564/1134] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:09:20 2022] POST /identity/v3/projects => generated 312 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:09:22.348797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb99ff7-6459-425e-8c26-cfd92d0cb2f0 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:22.349317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb99ff7-6459-425e-8c26-cfd92d0cb2f0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:22.349317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb99ff7-6459-425e-8c26-cfd92d0cb2f0 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:22.350695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 571/1135] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:22.355763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535d6b-5662-4771-b21a-bef35c6a9e8c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:22.356151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535d6b-5662-4771-b21a-bef35c6a9e8c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:22.356326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535d6b-5662-4771-b21a-bef35c6a9e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:22.374765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-92535d6b-5662-4771-b21a-bef35c6a9e8c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:22.375968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-92535d6b-5662-4771-b21a-bef35c6a9e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:22.440730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 565/1136] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:22 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:22.465736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb99ff7-6459-425e-8c26-cfd92d0cb2f0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:22.508779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:22.564568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb99ff7-6459-425e-8c26-cfd92d0cb2f0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:22.597798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe538c4f-c0da-4f78-a908-14bf07e2d289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:22.598430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538c4f-c0da-4f78-a908-14bf07e2d289 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:22.598710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538c4f-c0da-4f78-a908-14bf07e2d289 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:22.598983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538c4f-c0da-4f78-a908-14bf07e2d289 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:22.600132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe538c4f-c0da-4f78-a908-14bf07e2d289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:22.601588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe538c4f-c0da-4f78-a908-14bf07e2d289 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:22.603181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 572/1137] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:09:22 2022] GET /identity/v3/domains/default => generated 200 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:22.609442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535d6b-5662-4771-b21a-bef35c6a9e8c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:22.686100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:22.709127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535d6b-5662-4771-b21a-bef35c6a9e8c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:22.743138 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f086e-ac98-45b0-9729-d384739b6d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:22.743752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f086e-ac98-45b0-9729-d384739b6d0f None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:22.744036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f086e-ac98-45b0-9729-d384739b6d0f None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:22.744309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f086e-ac98-45b0-9729-d384739b6d0f None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:22.744996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f086e-ac98-45b0-9729-d384739b6d0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:22.746477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f086e-ac98-45b0-9729-d384739b6d0f None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:22.767623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 566/1138] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:09:22 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:09:24.807385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212c537-ad32-4672-9113-e5cb0d1d16fa None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:24.807943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212c537-ad32-4672-9113-e5cb0d1d16fa None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:24.807943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212c537-ad32-4672-9113-e5cb0d1d16fa None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:24.809265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 573/1139] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:24.815724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed46cbe-e6e5-4a33-a11b-6e33963f589a None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:24.816133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed46cbe-e6e5-4a33-a11b-6e33963f589a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:24.816271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed46cbe-e6e5-4a33-a11b-6e33963f589a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:24.834734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9ed46cbe-e6e5-4a33-a11b-6e33963f589a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:24.835916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9ed46cbe-e6e5-4a33-a11b-6e33963f589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:24.901215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 567/1140] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:24 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:24.925657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212c537-ad32-4672-9113-e5cb0d1d16fa None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:25.025558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212c537-ad32-4672-9113-e5cb0d1d16fa None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:25.035921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:25.059478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fe6b3-ecb1-4f69-9a39-cbf8b8dd5a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:25.060039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fe6b3-ecb1-4f69-9a39-cbf8b8dd5a10 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:25.060315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fe6b3-ecb1-4f69-9a39-cbf8b8dd5a10 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:25.060585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fe6b3-ecb1-4f69-9a39-cbf8b8dd5a10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:25.076302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fe6b3-ecb1-4f69-9a39-cbf8b8dd5a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-ce8fe6b3-ecb1-4f69-9a39-cbf8b8dd5a10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:25.081942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:25.083149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:25.084257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 574/1141] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:09:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:25.089256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed46cbe-e6e5-4a33-a11b-6e33963f589a None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:25.150053 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:25.186354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed46cbe-e6e5-4a33-a11b-6e33963f589a None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:25.219302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:25.219902 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:25.220175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:25.220442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:25.221139 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:25.221460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:25.222818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:25.230485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 568/1142] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:09:25 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:25.236963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fe6b3-ecb1-4f69-9a39-cbf8b8dd5a10 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:25.334347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fe6b3-ecb1-4f69-9a39-cbf8b8dd5a10 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:25.369131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4ea9-365a-4bbb-8f39-00068d850465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:25.369794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4ea9-365a-4bbb-8f39-00068d850465 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:25.370075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4ea9-365a-4bbb-8f39-00068d850465 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:25.370346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4ea9-365a-4bbb-8f39-00068d850465 None admin] PATH_INFO: `/v3/users/0229dd0181b64a41867310c3cadc3bc2` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:25.384073 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4ea9-365a-4bbb-8f39-00068d850465 None admin] RBAC: Authorizing `identity:get_user(user_id=0229dd0181b64a41867310c3cadc3bc2)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:25.385561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4ea9-365a-4bbb-8f39-00068d850465 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:25.387161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 575/1143] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:25 2022] GET /identity/v3/users/0229dd0181b64a41867310c3cadc3bc2 => generated 287 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:25.393593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:25.491534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7c72d-52a3-43de-add2-ea29def1b6bb None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:25.516903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:25.525754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1fce1-a749-4962-ab69-6398023a4885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:25.526363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1fce1-a749-4962-ab69-6398023a4885 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:25.526642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1fce1-a749-4962-ab69-6398023a4885 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:25.526912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1fce1-a749-4962-ab69-6398023a4885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:25.592153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1fce1-a749-4962-ab69-6398023a4885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:25.594290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1fce1-a749-4962-ab69-6398023a4885 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:25.698259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 569/1144] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:25 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:25.704935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4ea9-365a-4bbb-8f39-00068d850465 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:25.734005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:25.805078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4ea9-365a-4bbb-8f39-00068d850465 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:25.841225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ceefab-d0c7-47ca-b026-6f4dea2fe51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:25.842046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ceefab-d0c7-47ca-b026-6f4dea2fe51c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:25.842216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ceefab-d0c7-47ca-b026-6f4dea2fe51c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:25.842404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ceefab-d0c7-47ca-b026-6f4dea2fe51c None admin] PATH_INFO: `/v3/projects/c15de16e53a846e58b0f4f1c11f1adc5` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:25.843830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ceefab-d0c7-47ca-b026-6f4dea2fe51c None admin] RBAC: Authorizing `identity:get_project(project_id=c15de16e53a846e58b0f4f1c11f1adc5)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:25.845271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ceefab-d0c7-47ca-b026-6f4dea2fe51c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:25.847120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 576/1145] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:25 2022] GET /identity/v3/projects/c15de16e53a846e58b0f4f1c11f1adc5 => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:25.853239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1fce1-a749-4962-ab69-6398023a4885 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:25.882878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:25.961204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1fce1-a749-4962-ab69-6398023a4885 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:26.004045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5a154-5f33-493a-851c-7b30076dc597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:26.004671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a154-5f33-493a-851c-7b30076dc597 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:26.004967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a154-5f33-493a-851c-7b30076dc597 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:26.005263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a154-5f33-493a-851c-7b30076dc597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:26.006168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5a154-5f33-493a-851c-7b30076dc597 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=c15de16e53a846e58b0f4f1c11f1adc5, user.id=0229dd0181b64a41867310c3cadc3bc2) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:26.006716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5a154-5f33-493a-851c-7b30076dc597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:26.008671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5a154-5f33-493a-851c-7b30076dc597 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:26.016240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 570/1146] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:25 2022] GET /identity/v3/role_assignments?user.id=0229dd0181b64a41867310c3cadc3bc2&scope.project.id=c15de16e53a846e58b0f4f1c11f1adc5&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:28.022856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f82443-7cbc-4b8f-9496-404644dbb39b None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:28.023395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f82443-7cbc-4b8f-9496-404644dbb39b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:28.023824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f82443-7cbc-4b8f-9496-404644dbb39b None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:28.026360 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 577/1147] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:28 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:28.031799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66f7b4-7cfd-4ff5-99f8-4e2e18c8153e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:28.032222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66f7b4-7cfd-4ff5-99f8-4e2e18c8153e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:28.032480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66f7b4-7cfd-4ff5-99f8-4e2e18c8153e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:28.052419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-4e66f7b4-7cfd-4ff5-99f8-4e2e18c8153e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:28.053650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4e66f7b4-7cfd-4ff5-99f8-4e2e18c8153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:28.119139 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 571/1148] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:28 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:28.142193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f82443-7cbc-4b8f-9496-404644dbb39b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:28.234748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:28.241696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f82443-7cbc-4b8f-9496-404644dbb39b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:28.275398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b128dc-0432-49ad-b3ca-26ed294e6bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:28.275969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b128dc-0432-49ad-b3ca-26ed294e6bad None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:28.276249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b128dc-0432-49ad-b3ca-26ed294e6bad None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:28.276521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b128dc-0432-49ad-b3ca-26ed294e6bad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:28.286407 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b128dc-0432-49ad-b3ca-26ed294e6bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-63b128dc-0432-49ad-b3ca-26ed294e6bad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:28.291261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:28.292334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:28.293206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 578/1149] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:28.298629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66f7b4-7cfd-4ff5-99f8-4e2e18c8153e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:28.319951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:28.396746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66f7b4-7cfd-4ff5-99f8-4e2e18c8153e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:28.429022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:28.429655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:28.429939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:28.430213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:28.430915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:28.431215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:28.432776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:28.441649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 572/1150] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:09:28 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:28.447544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b128dc-0432-49ad-b3ca-26ed294e6bad None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:28.546081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b128dc-0432-49ad-b3ca-26ed294e6bad None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:28.579848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384df6a-7f23-42ae-aed6-d4018d3c79d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:28.580472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384df6a-7f23-42ae-aed6-d4018d3c79d3 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:28.580753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384df6a-7f23-42ae-aed6-d4018d3c79d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:28.581026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384df6a-7f23-42ae-aed6-d4018d3c79d3 None admin] PATH_INFO: `/v3/users/0229dd0181b64a41867310c3cadc3bc2` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:28.582211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384df6a-7f23-42ae-aed6-d4018d3c79d3 None admin] RBAC: Authorizing `identity:get_user(user_id=0229dd0181b64a41867310c3cadc3bc2)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:28.583665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384df6a-7f23-42ae-aed6-d4018d3c79d3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:28.585241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 579/1151] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:28 2022] GET /identity/v3/users/0229dd0181b64a41867310c3cadc3bc2 => generated 287 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:28.591506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:28.674129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:28.692540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073bef4-7030-4a44-82ca-de3fe00247dc None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:28.726415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a60135-cfc4-4e86-a933-ca2c689687d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:28.727045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60135-cfc4-4e86-a933-ca2c689687d8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:28.727327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60135-cfc4-4e86-a933-ca2c689687d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:28.727603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60135-cfc4-4e86-a933-ca2c689687d8 None admin] PATH_INFO: `/v3/projects/c15de16e53a846e58b0f4f1c11f1adc5` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:28.728760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a60135-cfc4-4e86-a933-ca2c689687d8 None admin] RBAC: Authorizing `identity:get_project(project_id=c15de16e53a846e58b0f4f1c11f1adc5)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:28.730224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a60135-cfc4-4e86-a933-ca2c689687d8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:28.731872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 573/1152] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:28 2022] GET /identity/v3/projects/c15de16e53a846e58b0f4f1c11f1adc5 => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:28.738278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384df6a-7f23-42ae-aed6-d4018d3c79d3 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:28.763111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:28.835677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384df6a-7f23-42ae-aed6-d4018d3c79d3 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:28.868863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136fc83-ba13-4510-9c46-7c32c271e8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:28.869537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136fc83-ba13-4510-9c46-7c32c271e8a1 None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:28.869814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136fc83-ba13-4510-9c46-7c32c271e8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:28.870082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136fc83-ba13-4510-9c46-7c32c271e8a1 None admin] PATH_INFO: `/v3/projects/c15de16e53a846e58b0f4f1c11f1adc5/users/0229dd0181b64a41867310c3cadc3bc2/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:28.871975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136fc83-ba13-4510-9c46-7c32c271e8a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c15de16e53a846e58b0f4f1c11f1adc5, user_id=0229dd0181b64a41867310c3cadc3bc2, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:28.873389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136fc83-ba13-4510-9c46-7c32c271e8a1 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:28.887166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 580/1153] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:09:28 2022] PUT /identity/v3/projects/c15de16e53a846e58b0f4f1c11f1adc5/users/0229dd0181b64a41867310c3cadc3bc2/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:09:30.836643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec933728-8aab-457f-b946-0fd049975683 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:30.837183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec933728-8aab-457f-b946-0fd049975683 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:30.837274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec933728-8aab-457f-b946-0fd049975683 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:30.838607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 574/1154] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:30.843589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a02130-e45c-4687-8004-46ff5cd2a740 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:30.844031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a02130-e45c-4687-8004-46ff5cd2a740 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:30.844131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a02130-e45c-4687-8004-46ff5cd2a740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:30.862396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-51a02130-e45c-4687-8004-46ff5cd2a740 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:30.863577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-51a02130-e45c-4687-8004-46ff5cd2a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:30.928654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 581/1155] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:30 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:30.951778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec933728-8aab-457f-b946-0fd049975683 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:31.049626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec933728-8aab-457f-b946-0fd049975683 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:31.082199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d8698-75ab-4b59-8029-af4503c88dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:31.082827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d8698-75ab-4b59-8029-af4503c88dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:31.083107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d8698-75ab-4b59-8029-af4503c88dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:31.083380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d8698-75ab-4b59-8029-af4503c88dd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:31.093460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d8698-75ab-4b59-8029-af4503c88dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-2c0d8698-75ab-4b59-8029-af4503c88dd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:31.098236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:31.099294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:31.100132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:31.101569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 575/1156] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:09:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:31.105401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a02130-e45c-4687-8004-46ff5cd2a740 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:31.120875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:31.205359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a02130-e45c-4687-8004-46ff5cd2a740 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:31.238158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:31.238756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:31.239028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:31.239290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:31.239984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:31.240278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:31.241690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:31.249497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 582/1157] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:09:31 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:31.255482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d8698-75ab-4b59-8029-af4503c88dd3 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:31.268530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:31.355236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d8698-75ab-4b59-8029-af4503c88dd3 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:31.387383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4b06f-ac03-46eb-9bd7-d3d3e3e50b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:31.388004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4b06f-ac03-46eb-9bd7-d3d3e3e50b57 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:31.388278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4b06f-ac03-46eb-9bd7-d3d3e3e50b57 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:31.388547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4b06f-ac03-46eb-9bd7-d3d3e3e50b57 None admin] PATH_INFO: `/v3/users/0229dd0181b64a41867310c3cadc3bc2` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:31.390040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4b06f-ac03-46eb-9bd7-d3d3e3e50b57 None admin] RBAC: Authorizing `identity:get_user(user_id=0229dd0181b64a41867310c3cadc3bc2)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:31.391464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4b06f-ac03-46eb-9bd7-d3d3e3e50b57 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:31.393040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 576/1158] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:31 2022] GET /identity/v3/users/0229dd0181b64a41867310c3cadc3bc2 => generated 287 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:31.398707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:31.473044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:31.495511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c55ca-76f3-4e8e-8be1-21618316b18a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:31.527092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23455423-ea8d-4fd4-bd58-124a1897e260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:31.527693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23455423-ea8d-4fd4-bd58-124a1897e260 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:31.527970 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23455423-ea8d-4fd4-bd58-124a1897e260 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:31.528237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23455423-ea8d-4fd4-bd58-124a1897e260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:31.592973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23455423-ea8d-4fd4-bd58-124a1897e260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:31.594414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23455423-ea8d-4fd4-bd58-124a1897e260 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:31.665919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:31.688632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 583/1159] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:31 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:31.694546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4b06f-ac03-46eb-9bd7-d3d3e3e50b57 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:31.773707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:31.796095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4b06f-ac03-46eb-9bd7-d3d3e3e50b57 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:31.834306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1512588-9f98-41cc-99f2-027cf0840053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:31.834996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1512588-9f98-41cc-99f2-027cf0840053 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:31.835273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1512588-9f98-41cc-99f2-027cf0840053 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:31.835542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1512588-9f98-41cc-99f2-027cf0840053 None admin] PATH_INFO: `/v3/projects/c15de16e53a846e58b0f4f1c11f1adc5` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:31.836675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1512588-9f98-41cc-99f2-027cf0840053 None admin] RBAC: Authorizing `identity:get_project(project_id=c15de16e53a846e58b0f4f1c11f1adc5)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:31.838164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1512588-9f98-41cc-99f2-027cf0840053 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:31.839830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 577/1160] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:31 2022] GET /identity/v3/projects/c15de16e53a846e58b0f4f1c11f1adc5 => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:31.846654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23455423-ea8d-4fd4-bd58-124a1897e260 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:31.943849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23455423-ea8d-4fd4-bd58-124a1897e260 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:31.976913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aff43a-22aa-4adc-a4d9-a7a4a80f4555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:31.977588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff43a-22aa-4adc-a4d9-a7a4a80f4555 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:31.977873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff43a-22aa-4adc-a4d9-a7a4a80f4555 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:31.978187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff43a-22aa-4adc-a4d9-a7a4a80f4555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:31.979003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aff43a-22aa-4adc-a4d9-a7a4a80f4555 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=c15de16e53a846e58b0f4f1c11f1adc5, user.id=0229dd0181b64a41867310c3cadc3bc2) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:31.979346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aff43a-22aa-4adc-a4d9-a7a4a80f4555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:31.980768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aff43a-22aa-4adc-a4d9-a7a4a80f4555 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:31.986573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 584/1161] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:31 2022] GET /identity/v3/role_assignments?user.id=0229dd0181b64a41867310c3cadc3bc2&scope.project.id=c15de16e53a846e58b0f4f1c11f1adc5&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 610 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:33.994588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca895d11-e1c1-43c2-b49e-071b3e5973f1 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:33.995119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca895d11-e1c1-43c2-b49e-071b3e5973f1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:33.995119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca895d11-e1c1-43c2-b49e-071b3e5973f1 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:33.996491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 578/1162] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:34.002321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dc10e-93b2-41f3-9f12-9271014e1083 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:34.002740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dc10e-93b2-41f3-9f12-9271014e1083 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:34.002847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dc10e-93b2-41f3-9f12-9271014e1083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:34.020880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-cc7dc10e-93b2-41f3-9f12-9271014e1083 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:34.022078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cc7dc10e-93b2-41f3-9f12-9271014e1083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:34.056730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:34.089756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 585/1163] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:34 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:34.113000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca895d11-e1c1-43c2-b49e-071b3e5973f1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:34.211768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca895d11-e1c1-43c2-b49e-071b3e5973f1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:34.245098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055b0d8-b526-4d7e-be32-2074da8261b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:34.245671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b0d8-b526-4d7e-be32-2074da8261b2 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:34.245954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b0d8-b526-4d7e-be32-2074da8261b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:34.246228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b0d8-b526-4d7e-be32-2074da8261b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:34.261242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:34.311695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055b0d8-b526-4d7e-be32-2074da8261b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:34.313144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055b0d8-b526-4d7e-be32-2074da8261b2 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:34.408098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 579/1164] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:34 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:34.414227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dc10e-93b2-41f3-9f12-9271014e1083 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:34.510266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dc10e-93b2-41f3-9f12-9271014e1083 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:34.543867 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b65d8a-3f34-4e08-864f-0036e91b52f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:34.544498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b65d8a-3f34-4e08-864f-0036e91b52f0 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:34.544781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b65d8a-3f34-4e08-864f-0036e91b52f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:34.545056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b65d8a-3f34-4e08-864f-0036e91b52f0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:34.552233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b65d8a-3f34-4e08-864f-0036e91b52f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:34.553705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b65d8a-3f34-4e08-864f-0036e91b52f0 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e3b65d8a-3f34-4e08-864f-0036e91b52f0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:34.565006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:34.566224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:34.567254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 586/1165] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:09:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:34.572071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055b0d8-b526-4d7e-be32-2074da8261b2 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:34.616762 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:34.668655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055b0d8-b526-4d7e-be32-2074da8261b2 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:34.700586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab1a66-2331-43b5-892e-9193b139e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:34.701188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1a66-2331-43b5-892e-9193b139e7c3 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:34.702087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1a66-2331-43b5-892e-9193b139e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:34.702366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1a66-2331-43b5-892e-9193b139e7c3 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:34.703042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab1a66-2331-43b5-892e-9193b139e7c3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:34.703339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab1a66-2331-43b5-892e-9193b139e7c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:34.704696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab1a66-2331-43b5-892e-9193b139e7c3 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:34.711562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 580/1166] 10.222.0.109 () {56 vars in 1120 bytes} [Thu Aug 4 22:09:34 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:36.710363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59e324-a924-47d3-a2f0-08a31688146e None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:36.710941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59e324-a924-47d3-a2f0-08a31688146e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:36.710941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59e324-a924-47d3-a2f0-08a31688146e None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:36.712292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 587/1167] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:36.718589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db197-a9f3-4d24-a759-029c61aec6b4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:36.719086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db197-a9f3-4d24-a759-029c61aec6b4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:36.719201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db197-a9f3-4d24-a759-029c61aec6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:36.737171 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-892db197-a9f3-4d24-a759-029c61aec6b4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:36.738386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-892db197-a9f3-4d24-a759-029c61aec6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:36.802279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 581/1168] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:36 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:36.825934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59e324-a924-47d3-a2f0-08a31688146e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:36.855896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:36.927701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59e324-a924-47d3-a2f0-08a31688146e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:36.961295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792feaa-84df-41a6-8d64-3848f7df3070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:36.961868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792feaa-84df-41a6-8d64-3848f7df3070 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:36.962149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792feaa-84df-41a6-8d64-3848f7df3070 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:36.962419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792feaa-84df-41a6-8d64-3848f7df3070 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:36.963125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792feaa-84df-41a6-8d64-3848f7df3070 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:36.964476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792feaa-84df-41a6-8d64-3848f7df3070 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:36.979654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 588/1169] 10.222.0.109 () {60 vars in 1143 bytes} [Thu Aug 4 22:09:36 2022] POST /identity/v3/domains => generated 268 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:09:39.017063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89d2bc-8755-4fb7-bc09-fb9c10af3ed7 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:39.017672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89d2bc-8755-4fb7-bc09-fb9c10af3ed7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:39.017672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89d2bc-8755-4fb7-bc09-fb9c10af3ed7 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:39.019017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 582/1170] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:39.024077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb459-7a5f-47bd-9021-2a08997607c0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:39.024512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb459-7a5f-47bd-9021-2a08997607c0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:39.024629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfb459-7a5f-47bd-9021-2a08997607c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:39.042367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-02bfb459-7a5f-47bd-9021-2a08997607c0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:39.043545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-02bfb459-7a5f-47bd-9021-2a08997607c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:39.110498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 589/1171] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:39 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:39.134150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89d2bc-8755-4fb7-bc09-fb9c10af3ed7 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:39.175915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:39.231378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89d2bc-8755-4fb7-bc09-fb9c10af3ed7 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:39.264967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ba959-4796-46a8-bb71-2d47a5eaf7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:39.265615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ba959-4796-46a8-bb71-2d47a5eaf7ff None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:39.265892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ba959-4796-46a8-bb71-2d47a5eaf7ff None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:39.266159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ba959-4796-46a8-bb71-2d47a5eaf7ff None admin] PATH_INFO: `/v3/domains/24821d5b89be4babaa8f36f4269b0e5f` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:39.275979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ba959-4796-46a8-bb71-2d47a5eaf7ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=24821d5b89be4babaa8f36f4269b0e5f)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:39.277344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ba959-4796-46a8-bb71-2d47a5eaf7ff None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:39.279060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 583/1172] 10.222.0.109 () {56 vars in 1188 bytes} [Thu Aug 4 22:09:39 2022] GET /identity/v3/domains/24821d5b89be4babaa8f36f4269b0e5f => generated 268 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:39.284919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfb459-7a5f-47bd-9021-2a08997607c0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:39.306010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:39.382636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfb459-7a5f-47bd-9021-2a08997607c0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:39.419665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e1e60-651a-4efd-94e0-41f338197067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:39.421171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1e60-651a-4efd-94e0-41f338197067 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:39.421476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1e60-651a-4efd-94e0-41f338197067 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:39.421748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1e60-651a-4efd-94e0-41f338197067 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:39.422437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e1e60-651a-4efd-94e0-41f338197067 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:39.423854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e1e60-651a-4efd-94e0-41f338197067 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:39.450588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 590/1173] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:09:39 2022] POST /identity/v3/projects => generated 362 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:09:41.511570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19d08b-0e36-40b6-8b16-6d9a79bab047 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:41.512071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19d08b-0e36-40b6-8b16-6d9a79bab047 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:41.512071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19d08b-0e36-40b6-8b16-6d9a79bab047 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:41.513484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 584/1174] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:41.518796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472244-84ce-456c-a654-8f935b7d56d7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:41.519223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472244-84ce-456c-a654-8f935b7d56d7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:41.519353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472244-84ce-456c-a654-8f935b7d56d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:41.537224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ae472244-84ce-456c-a654-8f935b7d56d7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:41.538453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ae472244-84ce-456c-a654-8f935b7d56d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:41.603250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 591/1175] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:41 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:09:41.628912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19d08b-0e36-40b6-8b16-6d9a79bab047 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:41.724968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:41.729108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19d08b-0e36-40b6-8b16-6d9a79bab047 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:41.762217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09c061-f8fc-4ba3-89f2-f660c480f13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:41.762784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09c061-f8fc-4ba3-89f2-f660c480f13d None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:41.763065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09c061-f8fc-4ba3-89f2-f660c480f13d None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:41.763450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09c061-f8fc-4ba3-89f2-f660c480f13d None admin] PATH_INFO: `/v3/domains/24821d5b89be4babaa8f36f4269b0e5f` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:41.764887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09c061-f8fc-4ba3-89f2-f660c480f13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=24821d5b89be4babaa8f36f4269b0e5f)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:41.766348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09c061-f8fc-4ba3-89f2-f660c480f13d None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:41.767927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 585/1176] 10.222.0.109 () {56 vars in 1188 bytes} [Thu Aug 4 22:09:41 2022] GET /identity/v3/domains/24821d5b89be4babaa8f36f4269b0e5f => generated 268 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:41.773689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae472244-84ce-456c-a654-8f935b7d56d7 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:41.775867 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:41.872637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae472244-84ce-456c-a654-8f935b7d56d7 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:41.906872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363ba73-8f7d-4b2e-aab3-893ee56e1125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:41.907477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ba73-8f7d-4b2e-aab3-893ee56e1125 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:41.907753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ba73-8f7d-4b2e-aab3-893ee56e1125 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:41.908028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ba73-8f7d-4b2e-aab3-893ee56e1125 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:41.908700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363ba73-8f7d-4b2e-aab3-893ee56e1125 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:41.910147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363ba73-8f7d-4b2e-aab3-893ee56e1125 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:41.930886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 592/1177] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:09:41 2022] POST /identity/v3/users => generated 312 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:09:43.936762 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fbc6c-1651-418d-9299-3873340b8e02 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:43.937337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fbc6c-1651-418d-9299-3873340b8e02 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:43.937337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fbc6c-1651-418d-9299-3873340b8e02 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:43.938669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 586/1178] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:43.943897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99401-8422-497f-96ca-3d7570ed0ffc None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:43.944339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99401-8422-497f-96ca-3d7570ed0ffc None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:43.944435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99401-8422-497f-96ca-3d7570ed0ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:43.963087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f0c99401-8422-497f-96ca-3d7570ed0ffc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:43.964265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f0c99401-8422-497f-96ca-3d7570ed0ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:44.016502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:44.031730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 593/1179] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:43 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:44.054675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fbc6c-1651-418d-9299-3873340b8e02 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:44.152239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fbc6c-1651-418d-9299-3873340b8e02 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:44.186446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312a7d-9ffe-4a49-b2e9-f8b420ad3b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:44.187141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312a7d-9ffe-4a49-b2e9-f8b420ad3b06 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:44.187344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312a7d-9ffe-4a49-b2e9-f8b420ad3b06 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:44.187611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312a7d-9ffe-4a49-b2e9-f8b420ad3b06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:44.197338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c312a7d-9ffe-4a49-b2e9-f8b420ad3b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-6c312a7d-9ffe-4a49-b2e9-f8b420ad3b06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:44.202460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:44.203574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:44.204615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 587/1180] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:09:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:44.209978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c99401-8422-497f-96ca-3d7570ed0ffc None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:44.306818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c99401-8422-497f-96ca-3d7570ed0ffc None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:44.339082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:44.339688 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:44.339961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:44.340226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:44.340947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:44.341843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:44.343185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:44.350922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 594/1181] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:09:44 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:44.356556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312a7d-9ffe-4a49-b2e9-f8b420ad3b06 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:44.383846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:44.453444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312a7d-9ffe-4a49-b2e9-f8b420ad3b06 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:44.485556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed65552-b3b4-43bc-ba96-7938e9f72170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:44.486173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed65552-b3b4-43bc-ba96-7938e9f72170 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:44.486449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed65552-b3b4-43bc-ba96-7938e9f72170 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:44.486720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed65552-b3b4-43bc-ba96-7938e9f72170 None admin] PATH_INFO: `/v3/users/c50431bd12a04d189aa842aeb0a7fb18` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:44.499184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed65552-b3b4-43bc-ba96-7938e9f72170 None admin] RBAC: Authorizing `identity:get_user(user_id=c50431bd12a04d189aa842aeb0a7fb18)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:44.500615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed65552-b3b4-43bc-ba96-7938e9f72170 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:44.502256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 588/1182] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:44 2022] GET /identity/v3/users/c50431bd12a04d189aa842aeb0a7fb18 => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:44.507979 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:44.558306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:44.609274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a2c00-f690-4a6b-97ab-8a7f5a888a35 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:44.642717 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63f9dd-d271-4d14-9015-4ca96f53159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:44.643328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63f9dd-d271-4d14-9015-4ca96f53159e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:44.643608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63f9dd-d271-4d14-9015-4ca96f53159e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:44.643883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63f9dd-d271-4d14-9015-4ca96f53159e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:44.710365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63f9dd-d271-4d14-9015-4ca96f53159e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:44.711742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63f9dd-d271-4d14-9015-4ca96f53159e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:44.758989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:44.809146 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 595/1183] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:44 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:44.815650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed65552-b3b4-43bc-ba96-7938e9f72170 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:44.898349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:44.912711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed65552-b3b4-43bc-ba96-7938e9f72170 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:44.945343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd9fee-b300-4505-8bb7-88882efe7651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:44.945988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd9fee-b300-4505-8bb7-88882efe7651 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:44.946267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd9fee-b300-4505-8bb7-88882efe7651 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:44.946534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd9fee-b300-4505-8bb7-88882efe7651 None admin] PATH_INFO: `/v3/projects/942d1bd40f514221b2fbb79997a37f83` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:44.947660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd9fee-b300-4505-8bb7-88882efe7651 None admin] RBAC: Authorizing `identity:get_project(project_id=942d1bd40f514221b2fbb79997a37f83)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:44.949078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd9fee-b300-4505-8bb7-88882efe7651 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:44.951347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 589/1184] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:44 2022] GET /identity/v3/projects/942d1bd40f514221b2fbb79997a37f83 => generated 362 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:44.957498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63f9dd-d271-4d14-9015-4ca96f53159e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:45.052942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63f9dd-d271-4d14-9015-4ca96f53159e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:45.086150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6e1c2-93ed-4c37-9048-b6f0cdd0a9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:45.086752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6e1c2-93ed-4c37-9048-b6f0cdd0a9cc None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:45.087031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6e1c2-93ed-4c37-9048-b6f0cdd0a9cc None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:45.087303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6e1c2-93ed-4c37-9048-b6f0cdd0a9cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:45.088078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6e1c2-93ed-4c37-9048-b6f0cdd0a9cc None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=942d1bd40f514221b2fbb79997a37f83, user.id=c50431bd12a04d189aa842aeb0a7fb18) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:45.088385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6e1c2-93ed-4c37-9048-b6f0cdd0a9cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:45.089844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6e1c2-93ed-4c37-9048-b6f0cdd0a9cc None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:45.095296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 596/1185] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:44 2022] GET /identity/v3/role_assignments?user.id=c50431bd12a04d189aa842aeb0a7fb18&scope.project.id=942d1bd40f514221b2fbb79997a37f83&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 260 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:47.040573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced30d-8c33-46a8-ac2e-3c442de68280 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:47.041231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced30d-8c33-46a8-ac2e-3c442de68280 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:47.041231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced30d-8c33-46a8-ac2e-3c442de68280 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:47.042496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 590/1186] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:47.048170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d93a-56b5-4f0b-8594-74d65e167ff7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:47.048621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d93a-56b5-4f0b-8594-74d65e167ff7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:47.048734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d93a-56b5-4f0b-8594-74d65e167ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:47.066726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5612d93a-56b5-4f0b-8594-74d65e167ff7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:47.067907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5612d93a-56b5-4f0b-8594-74d65e167ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:47.077168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:47.136529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 597/1187] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:47 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:47.160446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fced30d-8c33-46a8-ac2e-3c442de68280 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:47.260457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fced30d-8c33-46a8-ac2e-3c442de68280 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:47.293815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b37b7f-7f14-45b9-95f5-271f8c2b0b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:47.295188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37b7f-7f14-45b9-95f5-271f8c2b0b72 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:47.295472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37b7f-7f14-45b9-95f5-271f8c2b0b72 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:47.295746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37b7f-7f14-45b9-95f5-271f8c2b0b72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:47.305485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b37b7f-7f14-45b9-95f5-271f8c2b0b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-d9b37b7f-7f14-45b9-95f5-271f8c2b0b72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:47.310577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:47.311775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:09:47.312722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 591/1188] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:09:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:47.317818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612d93a-56b5-4f0b-8594-74d65e167ff7 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:47.416216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612d93a-56b5-4f0b-8594-74d65e167ff7 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:47.448879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:47.449517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:47.449798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:47.450066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:47.451307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:47.451604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:47.452963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:47.460989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 598/1189] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:09:47 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:47.467225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b37b7f-7f14-45b9-95f5-271f8c2b0b72 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:47.483622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:47.569730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b37b7f-7f14-45b9-95f5-271f8c2b0b72 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:47.603032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2793a88-c31e-4c78-af55-22fe50fac805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:47.603640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2793a88-c31e-4c78-af55-22fe50fac805 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:47.603918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2793a88-c31e-4c78-af55-22fe50fac805 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:47.604190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2793a88-c31e-4c78-af55-22fe50fac805 None admin] PATH_INFO: `/v3/users/c50431bd12a04d189aa842aeb0a7fb18` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:47.605340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2793a88-c31e-4c78-af55-22fe50fac805 None admin] RBAC: Authorizing `identity:get_user(user_id=c50431bd12a04d189aa842aeb0a7fb18)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:47.606951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2793a88-c31e-4c78-af55-22fe50fac805 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:47.608558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 592/1190] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:47 2022] GET /identity/v3/users/c50431bd12a04d189aa842aeb0a7fb18 => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:47.614404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:47.623590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:47.713537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c104455-6a62-4a9d-a020-41db07af9d3d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:47.745810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c228e66-53d5-4b59-961a-cc0b38a3ab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:47.746441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c228e66-53d5-4b59-961a-cc0b38a3ab13 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:47.746722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c228e66-53d5-4b59-961a-cc0b38a3ab13 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:47.746993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c228e66-53d5-4b59-961a-cc0b38a3ab13 None admin] PATH_INFO: `/v3/projects/942d1bd40f514221b2fbb79997a37f83` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:47.749009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c228e66-53d5-4b59-961a-cc0b38a3ab13 None admin] RBAC: Authorizing `identity:get_project(project_id=942d1bd40f514221b2fbb79997a37f83)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:47.750479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c228e66-53d5-4b59-961a-cc0b38a3ab13 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:47.752127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 599/1191] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:47 2022] GET /identity/v3/projects/942d1bd40f514221b2fbb79997a37f83 => generated 362 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:47.758078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2793a88-c31e-4c78-af55-22fe50fac805 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:47.836854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:47.855146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2793a88-c31e-4c78-af55-22fe50fac805 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:47.889179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f171363-ad5c-41c4-a027-6cc68229d766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:47.889878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f171363-ad5c-41c4-a027-6cc68229d766 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:47.890162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f171363-ad5c-41c4-a027-6cc68229d766 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:47.890435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f171363-ad5c-41c4-a027-6cc68229d766 None admin] PATH_INFO: `/v3/projects/942d1bd40f514221b2fbb79997a37f83/users/c50431bd12a04d189aa842aeb0a7fb18/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:47.892372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f171363-ad5c-41c4-a027-6cc68229d766 None admin] RBAC: Authorizing `identity:create_grant(project_id=942d1bd40f514221b2fbb79997a37f83, user_id=c50431bd12a04d189aa842aeb0a7fb18, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:47.893845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f171363-ad5c-41c4-a027-6cc68229d766 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:47.906707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 593/1192] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:09:47 2022] PUT /identity/v3/projects/942d1bd40f514221b2fbb79997a37f83/users/c50431bd12a04d189aa842aeb0a7fb18/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:09:49.894579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0763-e20a-4151-a943-ab39ab91224f None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:49.895117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0763-e20a-4151-a943-ab39ab91224f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:49.895117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0763-e20a-4151-a943-ab39ab91224f None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:49.896476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 600/1193] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:09:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:09:49.901570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21c5a2-ea56-4fc5-9d3a-7ac6f9c3d86e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:49.901973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21c5a2-ea56-4fc5-9d3a-7ac6f9c3d86e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:49.902119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21c5a2-ea56-4fc5-9d3a-7ac6f9c3d86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:49.920362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7e21c5a2-ea56-4fc5-9d3a-7ac6f9c3d86e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:09:49.921565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7e21c5a2-ea56-4fc5-9d3a-7ac6f9c3d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:09:49.985138 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 594/1194] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:09:49 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:09:50.008061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d0763-e20a-4151-a943-ab39ab91224f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:50.080697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:50.105931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d0763-e20a-4151-a943-ab39ab91224f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:50.138594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df832d-3aab-415d-b42b-7f16bfbbf99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:50.139464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df832d-3aab-415d-b42b-7f16bfbbf99a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:50.139738 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df832d-3aab-415d-b42b-7f16bfbbf99a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:50.140004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df832d-3aab-415d-b42b-7f16bfbbf99a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:50.150119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df832d-3aab-415d-b42b-7f16bfbbf99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-c6df832d-3aab-415d-b42b-7f16bfbbf99a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:09:50.155373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:09:50.156500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:09:50.157727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 601/1195] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:09:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:09:50.162479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21c5a2-ea56-4fc5-9d3a-7ac6f9c3d86e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:50.236634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:50.266639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21c5a2-ea56-4fc5-9d3a-7ac6f9c3d86e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:50.300466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:50.301071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:50.301353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:50.301657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:50.302365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:50.302670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:50.304048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:50.311908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 595/1196] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:09:50 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:50.317693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df832d-3aab-415d-b42b-7f16bfbbf99a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:50.416706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df832d-3aab-415d-b42b-7f16bfbbf99a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:50.448920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9029cb7-8d0a-4baf-91cf-a663ea5bb9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:50.449579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9029cb7-8d0a-4baf-91cf-a663ea5bb9cd None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:50.449869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9029cb7-8d0a-4baf-91cf-a663ea5bb9cd None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:50.450798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9029cb7-8d0a-4baf-91cf-a663ea5bb9cd None admin] PATH_INFO: `/v3/users/c50431bd12a04d189aa842aeb0a7fb18` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:50.452417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9029cb7-8d0a-4baf-91cf-a663ea5bb9cd None admin] RBAC: Authorizing `identity:get_user(user_id=c50431bd12a04d189aa842aeb0a7fb18)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:50.453881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9029cb7-8d0a-4baf-91cf-a663ea5bb9cd None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:50.455471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 602/1197] 10.222.0.109 () {56 vars in 1182 bytes} [Thu Aug 4 22:09:50 2022] GET /identity/v3/users/c50431bd12a04d189aa842aeb0a7fb18 => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:50.461136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:50.562683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea69425-1f96-4673-bd86-20d56f941e36 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:50.596061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e288b1-d535-4fe9-be5d-634b2906bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:50.596703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e288b1-d535-4fe9-be5d-634b2906bf92 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:50.597008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e288b1-d535-4fe9-be5d-634b2906bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:50.597283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e288b1-d535-4fe9-be5d-634b2906bf92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:50.599956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:50.664618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e288b1-d535-4fe9-be5d-634b2906bf92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:50.666187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e288b1-d535-4fe9-be5d-634b2906bf92 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:50.761013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 596/1198] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:09:50 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:09:50.767892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9029cb7-8d0a-4baf-91cf-a663ea5bb9cd None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:50.770764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:50.867185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9029cb7-8d0a-4baf-91cf-a663ea5bb9cd None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:50.900166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5678f-7b92-4055-9e8e-b5788bae701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:50.900793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5678f-7b92-4055-9e8e-b5788bae701c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:50.901072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5678f-7b92-4055-9e8e-b5788bae701c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:50.901343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5678f-7b92-4055-9e8e-b5788bae701c None admin] PATH_INFO: `/v3/projects/942d1bd40f514221b2fbb79997a37f83` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:50.902519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5678f-7b92-4055-9e8e-b5788bae701c None admin] RBAC: Authorizing `identity:get_project(project_id=942d1bd40f514221b2fbb79997a37f83)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:50.904253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5678f-7b92-4055-9e8e-b5788bae701c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:50.905949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 603/1199] 10.222.0.109 () {56 vars in 1191 bytes} [Thu Aug 4 22:09:50 2022] GET /identity/v3/projects/942d1bd40f514221b2fbb79997a37f83 => generated 362 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:09:50.912095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e288b1-d535-4fe9-be5d-634b2906bf92 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:09:50.944264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:09:51.007584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e288b1-d535-4fe9-be5d-634b2906bf92 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:09:51.040152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5d95d-6d41-4780-88ef-35019682521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:09:51.040690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5d95d-6d41-4780-88ef-35019682521b None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:09:51.040964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5d95d-6d41-4780-88ef-35019682521b None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:09:51.041231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5d95d-6d41-4780-88ef-35019682521b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:09:51.042028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5d95d-6d41-4780-88ef-35019682521b None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=942d1bd40f514221b2fbb79997a37f83, user.id=c50431bd12a04d189aa842aeb0a7fb18) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:09:51.042330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5d95d-6d41-4780-88ef-35019682521b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:09:51.043756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5d95d-6d41-4780-88ef-35019682521b None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:09:51.049532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 597/1200] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:09:50 2022] GET /identity/v3/role_assignments?user.id=c50431bd12a04d189aa842aeb0a7fb18&scope.project.id=942d1bd40f514221b2fbb79997a37f83&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 610 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:00.896141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=65 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:01.046993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=51 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:46.073826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43944312-a31b-4319-9cbc-14654f2ca4b6 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:46.074427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43944312-a31b-4319-9cbc-14654f2ca4b6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:46.074427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43944312-a31b-4319-9cbc-14654f2ca4b6 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:46.075933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 604/1201] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:10:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:10:46.081984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a716b-5e6f-4ae3-a491-fd6e61bcc0e5 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:46.082525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a716b-5e6f-4ae3-a491-fd6e61bcc0e5 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:46.082889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a716b-5e6f-4ae3-a491-fd6e61bcc0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:46.112791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8b1a716b-5e6f-4ae3-a491-fd6e61bcc0e5 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:10:46.113999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8b1a716b-5e6f-4ae3-a491-fd6e61bcc0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:10:46.200239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 598/1202] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:10:46 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:10:46.230435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43944312-a31b-4319-9cbc-14654f2ca4b6 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:46.358597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43944312-a31b-4319-9cbc-14654f2ca4b6 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:46.396098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:46.396751 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:46.397072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:46.397295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:46.416989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:46.418808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:10:46.437046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:46.443764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:46.444976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:10:46.444976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:10:46.444976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:46.444976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:10:46.444976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 605/1203] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:46.447573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a716b-5e6f-4ae3-a491-fd6e61bcc0e5 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:46.595492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a716b-5e6f-4ae3-a491-fd6e61bcc0e5 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:46.627823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:46.637545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a828d-4b8b-41b6-af77-a26f785edf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:46.638309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a828d-4b8b-41b6-af77-a26f785edf60 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:46.638639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a828d-4b8b-41b6-af77-a26f785edf60 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:46.638836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a828d-4b8b-41b6-af77-a26f785edf60 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:46.639511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a828d-4b8b-41b6-af77-a26f785edf60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:46.639842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a828d-4b8b-41b6-af77-a26f785edf60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:46.641217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a828d-4b8b-41b6-af77-a26f785edf60 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:46.659110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 599/1204] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:10:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:10:46.661190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:46.721386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:46.791304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4c67a-5c09-4ea8-8843-2750fbe0cfb2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:46.842140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa3154-afca-4c47-8eb5-cba66488cf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:46.842752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3154-afca-4c47-8eb5-cba66488cf0c None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:46.843056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3154-afca-4c47-8eb5-cba66488cf0c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:46.843396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa3154-afca-4c47-8eb5-cba66488cf0c None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:46.844314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa3154-afca-4c47-8eb5-cba66488cf0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:46.845921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa3154-afca-4c47-8eb5-cba66488cf0c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:46.869507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 606/1205] 10.222.0.109 () {60 vars in 1137 bytes} [Thu Aug 4 22:10:46 2022] POST /identity/v3/users => generated 252 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:10:49.282978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fa3f-969a-48d1-9769-4752b5791de6 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:49.283549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fa3f-969a-48d1-9769-4752b5791de6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:49.283549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fa3f-969a-48d1-9769-4752b5791de6 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:49.285327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 600/1206] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:10:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:10:49.292722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1b6e-b485-43e3-95fc-1a85bead4f60 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:49.293265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1b6e-b485-43e3-95fc-1a85bead4f60 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:49.293473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1b6e-b485-43e3-95fc-1a85bead4f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:49.311949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e07e1b6e-b485-43e3-95fc-1a85bead4f60 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:10:49.313131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e07e1b6e-b485-43e3-95fc-1a85bead4f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:10:49.399390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 607/1207] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:10:49 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:10:49.422992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792fa3f-969a-48d1-9769-4752b5791de6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:49.550378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792fa3f-969a-48d1-9769-4752b5791de6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:49.587616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bce51-9253-459d-94c5-e0bc6264ca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:49.588282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bce51-9253-459d-94c5-e0bc6264ca59 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:49.588532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bce51-9253-459d-94c5-e0bc6264ca59 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:49.588840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bce51-9253-459d-94c5-e0bc6264ca59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:49.603085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bce51-9253-459d-94c5-e0bc6264ca59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-8b6bce51-9253-459d-94c5-e0bc6264ca59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:10:49.608632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:49.609864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:10:49.610782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 601/1208] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:10:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:49.616092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e1b6e-b485-43e3-95fc-1a85bead4f60 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:49.659390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:49.728029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e1b6e-b485-43e3-95fc-1a85bead4f60 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:49.769598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:49.770199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:49.770521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:49.770746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:49.771451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:49.771758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:49.773116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:49.781852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 608/1209] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:10:49 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:49.787348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bce51-9253-459d-94c5-e0bc6264ca59 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:49.850584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:49.916582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bce51-9253-459d-94c5-e0bc6264ca59 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:49.955226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:49.955865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:49.956198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:49.956464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:49.969180 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:49.970770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:10:49.989098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:49.990259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:49.991134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:10:49.991134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:10:49.991134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:49.991134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:10:49.991134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 602/1210] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:49.997069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:50.112575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:50.115538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a43d9-4996-447d-b43a-84af87d86c20 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:50.154848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:50.155553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:50.155920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:50.156204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:50.156969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:50.157279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:50.158799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:50.171872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 609/1211] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:10:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:50.177371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:50.295039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94aeb8-92df-4eaf-ae98-df5f9a740ace None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:50.297595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:50.333383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:50.334071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:50.334391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:50.334663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] PATH_INFO: `/v3/users/placement` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:50.342809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:50.344246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:10:50.356713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:50.357971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:10:50.358977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 603/1212] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:50.365369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:50.472674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e248b-9b78-4540-9dcf-5d20b8689b03 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:50.512484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:50.513145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:50.513448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:50.513771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:50.514616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:50.514939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:50.516443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:50.536435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 610/1213] 10.222.0.109 () {56 vars in 1148 bytes} [Thu Aug 4 22:10:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:50.543484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:50.665798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a54d-d750-42bb-bf04-2c3312e6cdae None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:50.710421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be81bc-4bef-4eb9-a2b6-b0f5c3ed73f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:50.711770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be81bc-4bef-4eb9-a2b6-b0f5c3ed73f1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:50.712444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be81bc-4bef-4eb9-a2b6-b0f5c3ed73f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:50.712444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be81bc-4bef-4eb9-a2b6-b0f5c3ed73f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:50.750461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:50.803116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be81bc-4bef-4eb9-a2b6-b0f5c3ed73f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:50.805804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be81bc-4bef-4eb9-a2b6-b0f5c3ed73f1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:50.930158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 604/1214] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:10:50 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:10:50.937009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:50.963938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:51.076000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d35e0-784b-4b32-80e1-22051e344652 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:51.122293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:51.123020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:51.123311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:51.123582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:51.140497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:51.141732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:10:51.161074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:51.163085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:51.164211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:10:51.164211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:10:51.164211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:51.164211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:10:51.164211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 611/1215] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:51.168893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be81bc-4bef-4eb9-a2b6-b0f5c3ed73f1 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:51.239344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:51.289943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be81bc-4bef-4eb9-a2b6-b0f5c3ed73f1 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:51.330662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:51.330662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:51.330662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:51.330662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:51.331561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:51.331561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:51.331561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:51.343814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 605/1216] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:10:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:10:51.349482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:51.430584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:51.470105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaab4f-2485-4c10-85c1-dc7c7447177f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:51.510066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:51.511013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:51.511432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:51.511621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:51.519391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:51.520997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:10:51.533743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:51.534766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:10:51.535678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 612/1217] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:51.541760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:51.654389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5291af-3f56-4ef6-923f-a9c6390649f9 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:51.683332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:51.696810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38a659-4770-45f6-9a0f-f7e11014ff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:51.697471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a659-4770-45f6-9a0f-f7e11014ff23 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:51.697735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a659-4770-45f6-9a0f-f7e11014ff23 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:51.698004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a659-4770-45f6-9a0f-f7e11014ff23 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:51.698811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38a659-4770-45f6-9a0f-f7e11014ff23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:51.699121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38a659-4770-45f6-9a0f-f7e11014ff23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:51.700547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38a659-4770-45f6-9a0f-f7e11014ff23 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:51.716892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 606/1218] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:51.723183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:51.846041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a4172-30a8-4fc8-9d36-547636a5947a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:51.884313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:51.892652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b473d-892d-48df-bd49-8d4009eee65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:51.892847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b473d-892d-48df-bd49-8d4009eee65d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:51.893099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b473d-892d-48df-bd49-8d4009eee65d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:51.893387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b473d-892d-48df-bd49-8d4009eee65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:51.894267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b473d-892d-48df-bd49-8d4009eee65d None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=0583af73d78a40deb2ca3af755928cee) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:51.894615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b473d-892d-48df-bd49-8d4009eee65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:51.896333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b473d-892d-48df-bd49-8d4009eee65d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:51.903012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 613/1219] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:10:51 2022] GET /identity/v3/role_assignments?user.id=0583af73d78a40deb2ca3af755928cee&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:10:53.968385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998e05b-d584-4a85-90de-a55e29ebae49 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:53.968942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998e05b-d584-4a85-90de-a55e29ebae49 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:53.968942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998e05b-d584-4a85-90de-a55e29ebae49 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:53.970310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 607/1220] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:10:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:10:53.975334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b9e9-9da7-4204-941e-d0713657c538 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:53.975762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b9e9-9da7-4204-941e-d0713657c538 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:53.975888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b9e9-9da7-4204-941e-d0713657c538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:53.994549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9fd8b9e9-9da7-4204-941e-d0713657c538 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:10:53.995731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9fd8b9e9-9da7-4204-941e-d0713657c538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:10:54.069721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 614/1221] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:10:53 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:10:54.094003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998e05b-d584-4a85-90de-a55e29ebae49 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:54.215313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998e05b-d584-4a85-90de-a55e29ebae49 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:54.249337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22036c2b-ca30-4386-bfd8-d4f2a4fcff13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:54.249921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036c2b-ca30-4386-bfd8-d4f2a4fcff13 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:54.250196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036c2b-ca30-4386-bfd8-d4f2a4fcff13 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:54.250466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036c2b-ca30-4386-bfd8-d4f2a4fcff13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:54.260838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22036c2b-ca30-4386-bfd8-d4f2a4fcff13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-22036c2b-ca30-4386-bfd8-d4f2a4fcff13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:10:54.265706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:54.266847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:10:54.267702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 608/1222] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:10:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:54.273465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b9e9-9da7-4204-941e-d0713657c538 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:54.383502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b9e9-9da7-4204-941e-d0713657c538 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:54.408386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:54.421383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:54.422024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:54.422302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:54.422567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:54.423273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] RBAC: Adding query filter params (name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:54.423569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:54.424928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:54.433357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 615/1223] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:10:54 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:54.440827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22036c2b-ca30-4386-bfd8-d4f2a4fcff13 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:54.490550 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:54.553872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22036c2b-ca30-4386-bfd8-d4f2a4fcff13 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:54.592512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:54.593177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:54.593508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:54.593791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:54.605680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:54.607174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:10:54.622211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:54.623595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:54.625058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:10:54.625058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:10:54.625058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:54.625058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:10:54.625058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 609/1224] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:54.629316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:54.738915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6c518-d281-43b4-a312-6736743d0dcb None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:54.776466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:54.777088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:54.777371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:54.777677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:54.778362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:54.778665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:54.780025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:54.793190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 616/1225] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:10:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:54.799135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:54.927006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:54.935683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed3977-bb96-4e34-b0b7-db6d02e904e4 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:54.971014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:54.971669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:54.971919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:54.972192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] PATH_INFO: `/v3/users/placement` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:54.980059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:54.981639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:10:54.994252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:54.995235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:10:54.996182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 610/1226] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:55.001557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:55.065168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:55.121074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21feabd7-804c-49e1-a02f-91d79e298c5c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:55.161595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:55.162230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:55.162505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:55.162777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:55.163572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:55.164726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:55.166217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:55.190356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 617/1227] 10.222.0.109 () {56 vars in 1148 bytes} [Thu Aug 4 22:10:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:55.191514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:55.306215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cc892-1c7e-4af1-b50b-957b12fe1abb None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:55.345543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:55.346157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:55.346434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:55.346752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:55.357712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:55.359195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:10:55.373524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:55.374637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:55.375547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:10:55.375547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:10:55.375547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:55.375547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:10:55.375547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 611/1228] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:10:55.380344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:55.497502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c98a-6245-41e0-b2d4-60a2f77a4c65 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:55.502454 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:55.535952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:55.536652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:55.536915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:55.537181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:55.537903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:55.538199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:55.540146 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:55.552008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 618/1229] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:10:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:55.557995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:55.567931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:55.677784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6cd5-bcdc-43a5-8d01-a3463e5e1c39 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:55.720806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:55.721558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:55.721840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:55.722260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:55.729366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:55.730847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:10:55.745794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:55.747288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:10:55.748671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 612/1230] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:55.754591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:55.875115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e2991-4e33-4dcb-9d02-f30ef38bfac2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:55.914190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f3f46-eeaf-474a-958b-57c915ab79d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:55.914807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f3f46-eeaf-474a-958b-57c915ab79d7 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:55.915086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f3f46-eeaf-474a-958b-57c915ab79d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:55.915368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f3f46-eeaf-474a-958b-57c915ab79d7 None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:55.916389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f3f46-eeaf-474a-958b-57c915ab79d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:55.916810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f3f46-eeaf-474a-958b-57c915ab79d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:55.918665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f3f46-eeaf-474a-958b-57c915ab79d7 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:55.933574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 619/1231] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:10:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:55.940526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:56.016105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:56.060508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837de78-5494-4cf2-8c92-f8d2fd90b3e8 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:56.103880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24717865-4e22-4543-86ad-9c8f4b3398a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:56.104720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717865-4e22-4543-86ad-9c8f4b3398a4 None admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:56.105030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717865-4e22-4543-86ad-9c8f4b3398a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:56.105368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717865-4e22-4543-86ad-9c8f4b3398a4 None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/0583af73d78a40deb2ca3af755928cee/roles/7329827bdc90439cb95f7d079698b690` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:56.121991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24717865-4e22-4543-86ad-9c8f4b3398a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=0583af73d78a40deb2ca3af755928cee, role_id=7329827bdc90439cb95f7d079698b690)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:56.123599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24717865-4e22-4543-86ad-9c8f4b3398a4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:56.137185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 613/1232] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:10:55 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/0583af73d78a40deb2ca3af755928cee/roles/7329827bdc90439cb95f7d079698b690 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:10:58.407990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86387ec-583c-48e7-9b4e-15cda41a7db7 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:58.408517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86387ec-583c-48e7-9b4e-15cda41a7db7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:58.408517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86387ec-583c-48e7-9b4e-15cda41a7db7 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:58.410684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 620/1233] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:10:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:10:58.416009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143c9f-19c1-42fb-afbc-3415f5227e9f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:58.416443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143c9f-19c1-42fb-afbc-3415f5227e9f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:58.416573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143c9f-19c1-42fb-afbc-3415f5227e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:58.437326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-5b143c9f-19c1-42fb-afbc-3415f5227e9f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:10:58.438560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5b143c9f-19c1-42fb-afbc-3415f5227e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:10:58.520771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 614/1234] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:10:58 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:10:58.549349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86387ec-583c-48e7-9b4e-15cda41a7db7 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:58.576907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:58.666412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86387ec-583c-48e7-9b4e-15cda41a7db7 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:58.709567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f4834-3bec-4037-8d6d-6ef0d7a10319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:58.710232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4834-3bec-4037-8d6d-6ef0d7a10319 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:58.710520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4834-3bec-4037-8d6d-6ef0d7a10319 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:58.710802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4834-3bec-4037-8d6d-6ef0d7a10319 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:58.721014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f4834-3bec-4037-8d6d-6ef0d7a10319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-315f4834-3bec-4037-8d6d-6ef0d7a10319 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:10:58.725866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:58.728213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:10:58.729240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 621/1235] 10.222.0.109 () {56 vars in 1107 bytes} [Thu Aug 4 22:10:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:58.734760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b143c9f-19c1-42fb-afbc-3415f5227e9f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:58.782106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:58.840399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b143c9f-19c1-42fb-afbc-3415f5227e9f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:58.882136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:58.883317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:58.883594 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:58.883859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:58.884570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] RBAC: Adding query filter params (name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:58.884855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:58.886521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:58.894892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 615/1236] 10.222.0.109 () {56 vars in 1108 bytes} [Thu Aug 4 22:10:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:10:58.902506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f4834-3bec-4037-8d6d-6ef0d7a10319 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:58.990974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:59.008135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f4834-3bec-4037-8d6d-6ef0d7a10319 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:59.045454 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:59.046205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:59.046538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:59.046902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:59.058522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:59.059999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:10:59.075987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:59.077109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:59.078229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:10:59.078229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:10:59.078229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:10:59.078229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:10:59.078229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 622/1237] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:10:59.082974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:59.187800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416613dc-0051-4dbc-84a3-92a80f98edc6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:59.197302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:59.232596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:59.233648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:59.234097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:59.234409 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:59.235468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:59.235772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:59.238317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:59.253185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 616/1238] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:10:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:59.259617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:59.362871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba810784-4e90-46fc-8767-70e0b92c579f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:59.401274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:59.401938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:59.402211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:59.402475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:59.405276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:59.416155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:59.417928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:10:59.432114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:10:59.433315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:10:59.434384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:10:59.435100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 623/1239] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:10:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:10:59.439681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:59.579435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fc5c2-1e6d-430f-b9e2-dfc62bd5aa93 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:59.623845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:59.624485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:59.624756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:59.625033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:59.625866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:10:59.626174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:59.627749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:10:59.653830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 617/1240] 10.222.0.109 () {56 vars in 1148 bytes} [Thu Aug 4 22:10:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:10:59.654882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:10:59.774136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab6a4a-36c1-44ca-b933-d84893237251 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:10:59.811739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfc12e-3338-4a13-81d7-79c5328779c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:10:59.812361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfc12e-3338-4a13-81d7-79c5328779c4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:10:59.812639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfc12e-3338-4a13-81d7-79c5328779c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:10:59.812908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfc12e-3338-4a13-81d7-79c5328779c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:10:59.885150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:10:59.892236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbfc12e-3338-4a13-81d7-79c5328779c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:10:59.893822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbfc12e-3338-4a13-81d7-79c5328779c4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:00.009261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 624/1241] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:10:59 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:11:00.015821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:00.051002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:00.130678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b3c38-4d13-4ee4-b0d6-469474d8823a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:00.170118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:00.170755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:00.171076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:00.171366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:00.183342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:00.184803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:11:00.205977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:00.207175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:00.208117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:11:00.208117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:11:00.208117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:00.208117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:00.208117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 618/1242] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:11:00.213902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfc12e-3338-4a13-81d7-79c5328779c4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:00.317616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:00.324684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfc12e-3338-4a13-81d7-79c5328779c4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:00.362757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:00.363393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:00.363667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:00.363931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:00.364624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:00.364916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:00.366337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:00.378756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 625/1243] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:11:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:00.385231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:00.473458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:00.499858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16533c6-e46b-4f82-a940-f08fe0991740 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:00.539247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:00.539582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:00.539861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:00.540223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:00.547483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:00.549163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:11:00.564345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:00.565288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:00.566234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 619/1244] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:11:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:00.572665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:00.690122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dab422-2818-43fd-99b2-e55bda4f9b5c None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:00.728525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae16c8-8126-4e1a-9ce2-714d35268b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:00.729077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae16c8-8126-4e1a-9ce2-714d35268b4e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:00.729492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae16c8-8126-4e1a-9ce2-714d35268b4e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:00.729790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae16c8-8126-4e1a-9ce2-714d35268b4e None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:00.730553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae16c8-8126-4e1a-9ce2-714d35268b4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:00.730991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae16c8-8126-4e1a-9ce2-714d35268b4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:00.733997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae16c8-8126-4e1a-9ce2-714d35268b4e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:00.748268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 626/1245] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:11:00.754712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:00.874003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b17be7-24b3-49a7-99de-4134088eba2e None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:00.925331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3236ae3-bffd-40f9-b5b1-b65424733fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:00.926470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3236ae3-bffd-40f9-b5b1-b65424733fa1 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:00.927012 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3236ae3-bffd-40f9-b5b1-b65424733fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:00.927502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3236ae3-bffd-40f9-b5b1-b65424733fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:00.928647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3236ae3-bffd-40f9-b5b1-b65424733fa1 None admin] RBAC: Adding query filter params (role.id=7329827bdc90439cb95f7d079698b690, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=0583af73d78a40deb2ca3af755928cee) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:00.929058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3236ae3-bffd-40f9-b5b1-b65424733fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:00.931319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3236ae3-bffd-40f9-b5b1-b65424733fa1 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:00.938189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 620/1246] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:11:00 2022] GET /identity/v3/role_assignments?user.id=0583af73d78a40deb2ca3af755928cee&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=7329827bdc90439cb95f7d079698b690 => generated 610 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:03.021650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b430d-6975-4df2-9f99-20d965121c72 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:03.022212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b430d-6975-4df2-9f99-20d965121c72 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:03.022212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b430d-6975-4df2-9f99-20d965121c72 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:03.024094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 627/1247] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:11:03 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:11:03.029662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3822bc3-5087-4e80-9758-eafd8dc7ad3a None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:03.030184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3822bc3-5087-4e80-9758-eafd8dc7ad3a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:03.030184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3822bc3-5087-4e80-9758-eafd8dc7ad3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:03.036669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:03.051636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a3822bc3-5087-4e80-9758-eafd8dc7ad3a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:11:03.052966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a3822bc3-5087-4e80-9758-eafd8dc7ad3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:11:03.146742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 621/1248] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:11:03 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:11:03.173465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b430d-6975-4df2-9f99-20d965121c72 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:03.196986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:03.293975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b430d-6975-4df2-9f99-20d965121c72 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:03.333497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ded157-ca28-4a34-8582-4d6b878f1490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:03.334184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ded157-ca28-4a34-8582-4d6b878f1490 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:03.334488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ded157-ca28-4a34-8582-4d6b878f1490 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:03.334805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ded157-ca28-4a34-8582-4d6b878f1490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:03.348688 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ded157-ca28-4a34-8582-4d6b878f1490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-61ded157-ca28-4a34-8582-4d6b878f1490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:11:03.354534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:03.355641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:03.356620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 628/1249] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:03.364053 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3822bc3-5087-4e80-9758-eafd8dc7ad3a None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:03.479361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3822bc3-5087-4e80-9758-eafd8dc7ad3a None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:03.521254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:03.528597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:03.529228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:03.529576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:03.529832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:03.530549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:03.530852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:03.532305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:03.541952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 622/1250] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:11:03 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:11:03.548050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ded157-ca28-4a34-8582-4d6b878f1490 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:03.640116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:03.666198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ded157-ca28-4a34-8582-4d6b878f1490 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:03.704870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:03.705543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:03.705858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:03.706086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:03.718171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:03.719917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:11:03.736397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:03.737539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:03.738648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:11:03.738648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:11:03.738648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:03.738648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:03.738648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 629/1251] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:11:03.744941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:03.863178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943beb7-078c-46a5-a146-ee3ee34afa9c None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:03.905329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:03.906275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:03.906583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:03.907003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:03.907841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:03.908311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:03.909904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:03.925486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 623/1252] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:11:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:11:03.932988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:04.046418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9973e-f55c-4cca-96b7-77e416fd78f4 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:04.075788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:04.085596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:04.086419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:04.086874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:04.087251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:04.095676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:04.097148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:11:04.111221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:04.112476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:04.113737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 630/1253] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:11:04.119715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:04.188291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:04.239697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96091d-991d-46a1-b783-9f8845b22cea None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:04.277332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:04.277978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:04.278259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:04.278534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:04.279334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:04.279755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:04.281395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:04.301282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 624/1254] 10.222.0.109 () {56 vars in 1148 bytes} [Thu Aug 4 22:11:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:04.307683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:04.427070 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5ab39-7ffb-4d90-86e0-0d1c4aac5868 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:04.466357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85432463-8e5e-4fdb-b9c9-c4b5e1680a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:04.467046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85432463-8e5e-4fdb-b9c9-c4b5e1680a5a None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:04.467378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85432463-8e5e-4fdb-b9c9-c4b5e1680a5a None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:04.467741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85432463-8e5e-4fdb-b9c9-c4b5e1680a5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:04.534856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:04.551151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85432463-8e5e-4fdb-b9c9-c4b5e1680a5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:04.552710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85432463-8e5e-4fdb-b9c9-c4b5e1680a5a None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:04.669230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 631/1255] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:11:04 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:11:04.675694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:04.801634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2124d85-c767-4467-93d0-1c2ee9629ae6 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:04.819530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:04.848565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:04.849211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:04.849524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:04.849805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:04.863302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:04.864868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:11:04.884468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:04.886707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:04.887799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:11:04.887799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:11:04.887799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:04.887799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:04.887799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 625/1256] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:11:04.892143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85432463-8e5e-4fdb-b9c9-c4b5e1680a5a None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:04.995335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:05.012810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85432463-8e5e-4fdb-b9c9-c4b5e1680a5a None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:05.052296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:05.052986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:05.053113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:05.053388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:05.054116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:05.054525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:05.055972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:05.069447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 632/1257] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:11:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:11:05.075164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:05.189281 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a289fac-f5be-4b83-9cfe-3477a8e5cdad None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:05.232202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:05.232844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:05.233133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:05.233416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] PATH_INFO: `/v3/projects/service` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:05.243416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:05.245027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:11:05.260145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:05.261475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:05.262561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 626/1258] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:05.268008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:05.375384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1047d-12d3-4c29-bb56-6a172f04ae73 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:05.411966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cff2af-1234-48e4-95ba-ee1237b97fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:05.412625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff2af-1234-48e4-95ba-ee1237b97fdf None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:05.412906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff2af-1234-48e4-95ba-ee1237b97fdf None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:05.413180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff2af-1234-48e4-95ba-ee1237b97fdf None admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:05.414714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cff2af-1234-48e4-95ba-ee1237b97fdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:05.415025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cff2af-1234-48e4-95ba-ee1237b97fdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:05.416461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cff2af-1234-48e4-95ba-ee1237b97fdf None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:05.430799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 633/1259] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:05.437602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:05.468989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:05.554969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289236f1-6f74-4a63-a2af-09617cb9dafd None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:05.592018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be55f6b-e212-43e4-947c-0dc095c806cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:05.592645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55f6b-e212-43e4-947c-0dc095c806cd None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:05.592959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55f6b-e212-43e4-947c-0dc095c806cd None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:05.593261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55f6b-e212-43e4-947c-0dc095c806cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:05.594119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55f6b-e212-43e4-947c-0dc095c806cd None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=0583af73d78a40deb2ca3af755928cee) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:05.594462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55f6b-e212-43e4-947c-0dc095c806cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:05.595980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55f6b-e212-43e4-947c-0dc095c806cd None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:05.601777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 627/1260] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:11:05 2022] GET /identity/v3/role_assignments?user.id=0583af73d78a40deb2ca3af755928cee&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:07.667833 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9e873-981b-4585-b20b-dad78c1e1c32 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:07.668406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9e873-981b-4585-b20b-dad78c1e1c32 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:07.668406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9e873-981b-4585-b20b-dad78c1e1c32 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:07.669766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 634/1261] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:11:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:11:07.674892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f113f-41bf-4baa-bf2b-df5f359286ed None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:07.675284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f113f-41bf-4baa-bf2b-df5f359286ed None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:07.675447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f113f-41bf-4baa-bf2b-df5f359286ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:07.694735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-364f113f-41bf-4baa-bf2b-df5f359286ed None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:11:07.696447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-364f113f-41bf-4baa-bf2b-df5f359286ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:11:07.762746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 628/1262] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:11:07 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:11:07.785027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9e873-981b-4585-b20b-dad78c1e1c32 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:07.796932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:07.904258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9e873-981b-4585-b20b-dad78c1e1c32 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:07.937788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032168ce-a3c6-4095-bc34-6868487f21df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:07.938355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032168ce-a3c6-4095-bc34-6868487f21df None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:07.938632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032168ce-a3c6-4095-bc34-6868487f21df None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:07.938902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032168ce-a3c6-4095-bc34-6868487f21df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:07.949036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032168ce-a3c6-4095-bc34-6868487f21df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-032168ce-a3c6-4095-bc34-6868487f21df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:11:07.953958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:07.955021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:07.956282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 635/1263] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:11:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 22:11:07.961909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f113f-41bf-4baa-bf2b-df5f359286ed None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:07.980029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:08.058395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f113f-41bf-4baa-bf2b-df5f359286ed None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:08.091229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:08.091838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:08.092119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:08.092394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:08.093101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:08.093430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:08.094811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:08.105352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 629/1264] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:11:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:08.111704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032168ce-a3c6-4095-bc34-6868487f21df None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:08.177801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:08.209281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032168ce-a3c6-4095-bc34-6868487f21df None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:08.242885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:08.243498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:08.243772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:08.244037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:08.253645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:08.255070 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:11:08.269336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:08.270517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:08.271758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:11:08.271758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:11:08.271758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:08.271758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:08.271758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 636/1265] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:08.276923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:08.356114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:08.374963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3e4b-fc8a-411e-8255-dfb7bd443b20 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:08.407506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:08.408107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:08.408385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:08.408655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:08.409334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:08.409664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:08.411032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:08.422835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 630/1266] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:11:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:08.428312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:08.526950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ab59-6753-49b4-a6b3-b2aef6e3f26f None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:08.546541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:08.561164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:08.561820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:08.562104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:08.562377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:08.571007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:08.572438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:11:08.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:08.584512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:08.586474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 637/1267] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:08.590842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:08.685840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7834ce-8827-4c33-abf5-32012e9532af None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:08.718390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:08.718984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:08.719259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:08.719527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:08.720298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:08.720597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:08.722048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:08.730317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:08.738418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 631/1268] 10.222.0.109 () {56 vars in 1148 bytes} [Thu Aug 4 22:11:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:08.744414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:08.846002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29596b6e-61de-4332-b134-bf84daec89ba None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:08.882099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:08.883902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:08.884177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:08.884448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:08.895001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:08.896438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:11:08.910703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:08.911748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:08.913015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:11:08.913015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:11:08.913015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:08.913015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:08.913015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 638/1269] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:08.917695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:09.013420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccc18f-9c7d-45d2-bb9a-d6b16f94f933 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:09.045042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:09.045672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:09.045948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:09.046219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:09.046886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:09.047183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:09.048547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:09.059832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 632/1270] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:11:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:09.065453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:09.096644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:09.165344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e29a3-555b-4332-9f10-cdf276fe89f2 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:09.198400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:09.199032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:09.199312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:09.199585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:09.206112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:09.207547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:11:09.218865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:09.219909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:09.220831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 639/1271] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:11:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:09.225965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:09.283713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:09.324650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a5d6c-4993-4c84-82ef-e35cb55f5db4 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:09.356635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b2b7b-21ca-4af4-ba0c-deebfed63512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:09.357216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b2b7b-21ca-4af4-ba0c-deebfed63512 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:09.357482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b2b7b-21ca-4af4-ba0c-deebfed63512 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:09.357756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b2b7b-21ca-4af4-ba0c-deebfed63512 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:09.358490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b2b7b-21ca-4af4-ba0c-deebfed63512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:09.358790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b2b7b-21ca-4af4-ba0c-deebfed63512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:09.360207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b2b7b-21ca-4af4-ba0c-deebfed63512 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:09.373024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 633/1272] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:11:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:09.378926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:09.469132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:09.479039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a0381-a31f-41fa-b2c9-10c873edc64e None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:09.513011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e050146-f816-4958-a3a8-e322c8bed4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:09.513705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e050146-f816-4958-a3a8-e322c8bed4ef None admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:09.513987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e050146-f816-4958-a3a8-e322c8bed4ef None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:09.514261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e050146-f816-4958-a3a8-e322c8bed4ef None admin] PATH_INFO: `/v3/projects/01df3a6495b04e38be92a133494b85cb/users/0583af73d78a40deb2ca3af755928cee/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:09.516191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e050146-f816-4958-a3a8-e322c8bed4ef None admin] RBAC: Authorizing `identity:create_grant(project_id=01df3a6495b04e38be92a133494b85cb, user_id=0583af73d78a40deb2ca3af755928cee, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:09.517654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e050146-f816-4958-a3a8-e322c8bed4ef None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:09.531397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 640/1273] 10.222.0.109 () {58 vars in 1444 bytes} [Thu Aug 4 22:11:09 2022] PUT /identity/v3/projects/01df3a6495b04e38be92a133494b85cb/users/0583af73d78a40deb2ca3af755928cee/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:11:11.548065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac13dc-1b61-4b53-9412-85977356a060 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:11.548569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac13dc-1b61-4b53-9412-85977356a060 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:11.548569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac13dc-1b61-4b53-9412-85977356a060 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:11.549939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 634/1274] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:11:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:11:11.555664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476004a-fedf-4c4f-a5be-92278e5b7f06 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:11.556075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476004a-fedf-4c4f-a5be-92278e5b7f06 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:11.556213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476004a-fedf-4c4f-a5be-92278e5b7f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:11.574287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7476004a-fedf-4c4f-a5be-92278e5b7f06 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:11:11.575474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7476004a-fedf-4c4f-a5be-92278e5b7f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:11:11.640937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 641/1275] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:11:11 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:11:11.664310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac13dc-1b61-4b53-9412-85977356a060 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:11.761692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac13dc-1b61-4b53-9412-85977356a060 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:11.778770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:11.796217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd2069-db74-4823-b7ef-00e29299081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:11.796846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd2069-db74-4823-b7ef-00e29299081a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:11.797128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd2069-db74-4823-b7ef-00e29299081a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:11.797398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd2069-db74-4823-b7ef-00e29299081a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:11.807286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd2069-db74-4823-b7ef-00e29299081a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-bbcd2069-db74-4823-b7ef-00e29299081a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 22:11:11.812122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:11.813047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:11.813995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 635/1276] 10.222.0.109 () {56 vars in 1101 bytes} [Thu Aug 4 22:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:11.820451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7476004a-fedf-4c4f-a5be-92278e5b7f06 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:11.919341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:11.940466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7476004a-fedf-4c4f-a5be-92278e5b7f06 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:11.974993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:11.975614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:11.975893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:11.976528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:11.977240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:11.977579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:11.978943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:11.986812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 642/1277] 10.222.0.109 () {56 vars in 1104 bytes} [Thu Aug 4 22:11:11 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:11.993065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd2069-db74-4823-b7ef-00e29299081a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:12.088782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd2069-db74-4823-b7ef-00e29299081a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:12.123043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:12.123702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:12.123979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:12.124244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:12.133876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:12.135305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:11:12.149681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:12.150569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:12.151465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:11:12.151465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:11:12.151465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:12.151465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:12.151465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 636/1278] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:12.156625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:12.255438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5df001-aaf9-42bf-840e-022852b30ef5 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:12.287706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:12.288308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:12.289103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:12.289380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:12.290087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:12.290386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:12.291741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:12.303016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 643/1279] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:11:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:12.309356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:12.340457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:12.407224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe51f8-470b-4d33-a6a5-8f4dfbfdf250 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:12.439253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:12.439869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:12.440143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:12.440408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] PATH_INFO: `/v3/users/placement` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:12.447493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:12.448934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 22:11:12.459813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:12.461441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:12.462819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 637/1280] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:12.467427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:12.471222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:12.582953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf616199-3aee-4df3-af9b-01c5e3cc9c0d None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:12.616274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:12.617035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:12.617490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:12.617766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:12.618608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:12.618908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:12.620369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:12.636971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 644/1281] 10.222.0.109 () {56 vars in 1148 bytes} [Thu Aug 4 22:11:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:11:12.642813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:12.732204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:12.742004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b3818-055f-47e9-a6bb-63bd6921286a None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:12.774567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732070d-d00b-4cab-90bb-2b958f9202b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:12.775175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732070d-d00b-4cab-90bb-2b958f9202b7 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:12.775458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732070d-d00b-4cab-90bb-2b958f9202b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:12.775731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732070d-d00b-4cab-90bb-2b958f9202b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:12.848055 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732070d-d00b-4cab-90bb-2b958f9202b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:12.849634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732070d-d00b-4cab-90bb-2b958f9202b7 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:12.935792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:12.946532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 638/1282] 10.222.0.109 () {58 vars in 1287 bytes} [Thu Aug 4 22:11:12 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 22:11:12.952394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:13.013671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:13.051926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a46db-54d2-4109-8f3e-e8d21f9b3386 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:13.085294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:13.086622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:13.086880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:13.087154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:13.096904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:13.098376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 22:11:13.114527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:13.115568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:13.116838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 22:11:13.116838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 22:11:13.116838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 22:11:13.116838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:13.116838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 645/1283] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:13.121974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732070d-d00b-4cab-90bb-2b958f9202b7 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:13.217159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732070d-d00b-4cab-90bb-2b958f9202b7 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:13.249850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:13.250453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:13.250733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:13.251004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:13.251682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:13.251981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:13.253341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:13.264999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 639/1284] 10.222.0.109 () {56 vars in 1114 bytes} [Thu Aug 4 22:11:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:13.270538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:13.371067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012ce81-ca74-4704-8d96-4e17fed1f742 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:13.385181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:13.405638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:13.406269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:13.406548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:13.406821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:13.413186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:13.415184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 22:11:13.427029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:13.428074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: ERROR keystone.server.flask.application Aug 04 22:11:13.429929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 646/1285] 10.222.0.109 () {56 vars in 1116 bytes} [Thu Aug 4 22:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:13.434507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:13.482571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:13.530603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404f1f1-2925-46b2-96a2-fcfc24f5e966 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:13.562782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d54e3-acdd-4e83-bc25-163dc3dbbad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:13.563381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d54e3-acdd-4e83-bc25-163dc3dbbad4 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:13.563657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d54e3-acdd-4e83-bc25-163dc3dbbad4 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:13.563923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d54e3-acdd-4e83-bc25-163dc3dbbad4 None admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:13.564653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d54e3-acdd-4e83-bc25-163dc3dbbad4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:13.564951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d54e3-acdd-4e83-bc25-163dc3dbbad4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:13.566391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d54e3-acdd-4e83-bc25-163dc3dbbad4 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:13.578513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 640/1286] 10.222.0.109 () {56 vars in 1153 bytes} [Thu Aug 4 22:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:13.584347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:13.683662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0071763-1b7b-4728-ab4b-9c9164e52ec6 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:13.715604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ed56a-88d1-46b6-9cc1-9343a9e9a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:13.716208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ed56a-88d1-46b6-9cc1-9343a9e9a204 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:13.716487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ed56a-88d1-46b6-9cc1-9343a9e9a204 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:13.716759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ed56a-88d1-46b6-9cc1-9343a9e9a204 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:13.717642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ed56a-88d1-46b6-9cc1-9343a9e9a204 None admin] RBAC: Adding query filter params (role.id=68c4b28369a0474b83e1602e8e11c41c, scope.project.id=01df3a6495b04e38be92a133494b85cb, user.id=0583af73d78a40deb2ca3af755928cee) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:13.717958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ed56a-88d1-46b6-9cc1-9343a9e9a204 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:13.719374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ed56a-88d1-46b6-9cc1-9343a9e9a204 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:13.724978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 647/1287] 10.222.0.109 () {56 vars in 1379 bytes} [Thu Aug 4 22:11:13 2022] GET /identity/v3/role_assignments?user.id=0583af73d78a40deb2ca3af755928cee&scope.project.id=01df3a6495b04e38be92a133494b85cb&role.id=68c4b28369a0474b83e1602e8e11c41c => generated 610 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:15.793500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc9bca-e7fb-449e-af22-ca1dc70250d7 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:15.794063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc9bca-e7fb-449e-af22-ca1dc70250d7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:15.794063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc9bca-e7fb-449e-af22-ca1dc70250d7 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:15.795405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 641/1288] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:11:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:11:15.800597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09263838-81c0-4e17-8135-516ad773e2b8 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:15.801034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09263838-81c0-4e17-8135-516ad773e2b8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:15.801140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09263838-81c0-4e17-8135-516ad773e2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:15.819473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-09263838-81c0-4e17-8135-516ad773e2b8 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:11:15.821578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-09263838-81c0-4e17-8135-516ad773e2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:11:15.857259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:15.900097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 648/1289] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:11:15 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:11:15.922053 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc9bca-e7fb-449e-af22-ca1dc70250d7 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:16.022493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc9bca-e7fb-449e-af22-ca1dc70250d7 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:16.033106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:16.057579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:16.058146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:16.058423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:16.058694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:16.059306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:16.060686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:16.069287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:16.071217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:16.072300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 642/1290] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:11:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:16.077359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09263838-81c0-4e17-8135-516ad773e2b8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:16.174607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09263838-81c0-4e17-8135-516ad773e2b8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:16.208502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2716b0e-4140-493f-806d-6715c1551eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:16.209115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2716b0e-4140-493f-806d-6715c1551eff None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:16.209389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2716b0e-4140-493f-806d-6715c1551eff None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:16.209685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2716b0e-4140-493f-806d-6715c1551eff None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:16.210347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2716b0e-4140-493f-806d-6715c1551eff None admin] RBAC: Adding query filter params (name=placement) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:16.210669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2716b0e-4140-493f-806d-6715c1551eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:16.212205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2716b0e-4140-493f-806d-6715c1551eff None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:16.216846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 649/1291] 10.222.0.109 () {56 vars in 1121 bytes} [Thu Aug 4 22:11:16 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:16.222621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:16.322791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3ff57-ee8f-4693-8ea7-3835db5abb88 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:16.355386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a7489-7378-4d9c-8614-3fa36c58fc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:16.356023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7489-7378-4d9c-8614-3fa36c58fc9a None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:16.356301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7489-7378-4d9c-8614-3fa36c58fc9a None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:16.356568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7489-7378-4d9c-8614-3fa36c58fc9a None admin] PATH_INFO: `/v3/services` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:16.357233 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a7489-7378-4d9c-8614-3fa36c58fc9a None admin] RBAC: Adding query filter params (type=placement) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:16.357565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a7489-7378-4d9c-8614-3fa36c58fc9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:16.358925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a7489-7378-4d9c-8614-3fa36c58fc9a None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:16.363176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 643/1292] 10.222.0.109 () {56 vars in 1121 bytes} [Thu Aug 4 22:11:16 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:18.344495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432c2b-c6d2-4091-8851-b29f075d3400 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:18.345022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432c2b-c6d2-4091-8851-b29f075d3400 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:18.345022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432c2b-c6d2-4091-8851-b29f075d3400 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:18.346429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 650/1293] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:11:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:11:18.352092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96dca-9457-4f35-bbc9-909a9cfaf53f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:18.352529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96dca-9457-4f35-bbc9-909a9cfaf53f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:18.352721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96dca-9457-4f35-bbc9-909a9cfaf53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:18.371436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-efd96dca-9457-4f35-bbc9-909a9cfaf53f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:11:18.372541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-efd96dca-9457-4f35-bbc9-909a9cfaf53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:11:18.401396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:18.438100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 644/1294] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:11:18 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:11:18.460155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432c2b-c6d2-4091-8851-b29f075d3400 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:18.491719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:18.561255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432c2b-c6d2-4091-8851-b29f075d3400 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:18.596827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e617368-2151-4e6a-9d1a-fbb76efd9ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:18.597616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e617368-2151-4e6a-9d1a-fbb76efd9ed3 None admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:18.597785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e617368-2151-4e6a-9d1a-fbb76efd9ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:18.598010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e617368-2151-4e6a-9d1a-fbb76efd9ed3 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:18.598698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e617368-2151-4e6a-9d1a-fbb76efd9ed3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:18.600065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e617368-2151-4e6a-9d1a-fbb76efd9ed3 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:18.614343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 651/1295] 10.222.0.109 () {60 vars in 1146 bytes} [Thu Aug 4 22:11:18 2022] POST /identity/v3/services => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:11:20.602219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffac58e-be8e-471b-a917-216057aae1c0 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:20.602790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffac58e-be8e-471b-a917-216057aae1c0 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:20.602790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffac58e-be8e-471b-a917-216057aae1c0 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:20.604103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 645/1296] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:11:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:11:20.609207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad7ae-b912-4325-b657-b313381a3609 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:20.609679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad7ae-b912-4325-b657-b313381a3609 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:20.609781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad7ae-b912-4325-b657-b313381a3609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:20.629363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5d7ad7ae-b912-4325-b657-b313381a3609 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:11:20.630757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5d7ad7ae-b912-4325-b657-b313381a3609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:11:20.702632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 652/1297] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:11:20 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 22:11:20.725430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffac58e-be8e-471b-a917-216057aae1c0 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:20.824500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffac58e-be8e-471b-a917-216057aae1c0 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:20.864978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:20.865643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:20.865958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:20.866321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] PATH_INFO: `/v3/services/placement` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:20.867214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:20.868822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:20.879489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:20.880734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:20.882123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 646/1298] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:11:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:20.888055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad7ae-b912-4325-b657-b313381a3609 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:20.910139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:20.989606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad7ae-b912-4325-b657-b313381a3609 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:21.022451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:21.023551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:21.023820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:21.024081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:21.024733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:21.025026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:21.026413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:21.031321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 653/1299] 10.222.0.109 () {56 vars in 1121 bytes} [Thu Aug 4 22:11:20 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:21.037249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:21.068598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:21.136164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7e006-c70f-4627-be1d-f2118f30c1da None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:21.168844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41001be2-7af9-4425-8f23-7d014a8a8af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:21.169469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41001be2-7af9-4425-8f23-7d014a8a8af0 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:21.169749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41001be2-7af9-4425-8f23-7d014a8a8af0 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:21.170016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41001be2-7af9-4425-8f23-7d014a8a8af0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:21.170753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41001be2-7af9-4425-8f23-7d014a8a8af0 None admin] RBAC: Adding query filter params (interface=public, service_id=bc0d8278450e4a39944c7f77b86f90c9, region_id=RegionOne) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:21.171054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41001be2-7af9-4425-8f23-7d014a8a8af0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:21.172410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41001be2-7af9-4425-8f23-7d014a8a8af0 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:21.177216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 647/1300] 10.222.0.109 () {56 vars in 1256 bytes} [Thu Aug 4 22:11:21 2022] GET /identity/v3/endpoints?service_id=bc0d8278450e4a39944c7f77b86f90c9&interface=public®ion_id=RegionOne => generated 195 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:21.183574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:21.264392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:21.283222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573b9c-3516-4c64-9645-d47ad6fd1c51 None admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:21.316812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e02d7-8769-4bde-8999-b778355b21dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:21.317442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e02d7-8769-4bde-8999-b778355b21dd None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:21.317720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e02d7-8769-4bde-8999-b778355b21dd None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:21.317987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e02d7-8769-4bde-8999-b778355b21dd None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:21.318625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e02d7-8769-4bde-8999-b778355b21dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:21.320492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e02d7-8769-4bde-8999-b778355b21dd None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:21.327125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 654/1301] 10.222.0.109 () {56 vars in 1092 bytes} [Thu Aug 4 22:11:21 2022] GET /identity/v3/services => generated 2201 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:11:23.322950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8bed2-b1c3-4f35-af49-961e81c630e6 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:23.323469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8bed2-b1c3-4f35-af49-961e81c630e6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:23.323469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8bed2-b1c3-4f35-af49-961e81c630e6 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:23.324850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 648/1302] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:11:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:11:23.330137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b11a-b077-4d08-bd85-026b7af0f430 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:23.330568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b11a-b077-4d08-bd85-026b7af0f430 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:23.330671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b11a-b077-4d08-bd85-026b7af0f430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:23.348240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3397b11a-b077-4d08-bd85-026b7af0f430 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:11:23.349417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3397b11a-b077-4d08-bd85-026b7af0f430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:11:23.415447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 655/1303] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:11:23 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 22:11:23.438811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8bed2-b1c3-4f35-af49-961e81c630e6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:23.531620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:23.539325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8bed2-b1c3-4f35-af49-961e81c630e6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:23.572367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:23.572993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:23.573269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:23.573566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:23.574177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:23.575536 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 22:11:23.584065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 22:11:23.585308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application with Lock( Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self._enter() Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application return self.creator() Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application created_value = creator( Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: ERROR keystone.server.flask.application Aug 04 22:11:23.586324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 649/1304] 10.222.0.109 () {56 vars in 1122 bytes} [Thu Aug 4 22:11:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 22:11:23.591337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b11a-b077-4d08-bd85-026b7af0f430 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:23.664545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:23.687819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b11a-b077-4d08-bd85-026b7af0f430 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:23.721228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ce2b2-6d20-4dba-ab82-d236087329a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:23.721874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ce2b2-6d20-4dba-ab82-d236087329a2 None admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:23.722157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ce2b2-6d20-4dba-ab82-d236087329a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:23.722444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ce2b2-6d20-4dba-ab82-d236087329a2 None admin] PATH_INFO: `/v3/services` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:23.723130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ce2b2-6d20-4dba-ab82-d236087329a2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:11:23.723434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ce2b2-6d20-4dba-ab82-d236087329a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:23.724793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ce2b2-6d20-4dba-ab82-d236087329a2 None admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:23.730456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 656/1305] 10.222.0.109 () {56 vars in 1121 bytes} [Thu Aug 4 22:11:23 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:11:23.736336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:11:23.837230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b16f6f-36a9-4db0-b169-47e114f4ba09 None admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:11:23.879804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8243fe-631b-4d14-b7f7-eff6d6f5f674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:11:23.880605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8243fe-631b-4d14-b7f7-eff6d6f5f674 None admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:11:23.880932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8243fe-631b-4d14-b7f7-eff6d6f5f674 None admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:11:23.881424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8243fe-631b-4d14-b7f7-eff6d6f5f674 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:11:23.882617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8243fe-631b-4d14-b7f7-eff6d6f5f674 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:11:23.884351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8243fe-631b-4d14-b7f7-eff6d6f5f674 None admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:11:23.905995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 650/1306] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:11:23 2022] POST /identity/v3/endpoints => generated 333 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:11:33.727679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=30 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:11:33.901647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:27.094857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629d16-f971-4ce3-8c1c-2daea0adece5 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:27.095447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629d16-f971-4ce3-8c1c-2daea0adece5 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:27.095712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629d16-f971-4ce3-8c1c-2daea0adece5 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:27.097627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 657/1307] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:27.103268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2de3b-79cb-4e56-bcb9-a444183f5f72 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:27.103931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2de3b-79cb-4e56-bcb9-a444183f5f72 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:27.104344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2de3b-79cb-4e56-bcb9-a444183f5f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:27.183943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-c6b2de3b-79cb-4e56-bcb9-a444183f5f72 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:27.185297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c6b2de3b-79cb-4e56-bcb9-a444183f5f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:27.299307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 651/1308] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:27.353825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef190c1-b631-44c9-8de4-e56a1ecfc39a None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:27.354459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef190c1-b631-44c9-8de4-e56a1ecfc39a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:27.354885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef190c1-b631-44c9-8de4-e56a1ecfc39a None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:27.356851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 658/1309] 10.222.0.109 () {54 vars in 970 bytes} [Thu Aug 4 22:12:27 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:27.363351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c272f-65a7-4c06-98c8-a18f5cb08c94 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:27.363898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c272f-65a7-4c06-98c8-a18f5cb08c94 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:27.364349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c272f-65a7-4c06-98c8-a18f5cb08c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:27.536815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e78c272f-65a7-4c06-98c8-a18f5cb08c94 None None] MFA Rules not processed for user `5455c117e349480ba8b8c020b767bec5`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:27.538950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e78c272f-65a7-4c06-98c8-a18f5cb08c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:27.576379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:27.654896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 652/1310] 10.222.0.109 () {58 vars in 1067 bytes} [Thu Aug 4 22:12:27 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:27.666085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef190c1-b631-44c9-8de4-e56a1ecfc39a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:27.668160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9ef190c1-b631-44c9-8de4-e56a1ecfc39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:27.791818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef190c1-b631-44c9-8de4-e56a1ecfc39a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:27.884457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bc044-2400-49c9-bc18-1980e15edbb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:27.885189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bc044-2400-49c9-bc18-1980e15edbb7 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:27.885619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bc044-2400-49c9-bc18-1980e15edbb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:27.886036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bc044-2400-49c9-bc18-1980e15edbb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:27.888101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-392bc044-2400-49c9-bc18-1980e15edbb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:27.954390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:28.019192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bc044-2400-49c9-bc18-1980e15edbb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:28.020670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bc044-2400-49c9-bc18-1980e15edbb7 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:28.043111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 659/1311] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:27 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:30.446345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adde45d-afa7-4c25-850c-7182b3f7ae3c None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:30.446886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adde45d-afa7-4c25-850c-7182b3f7ae3c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:30.446886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adde45d-afa7-4c25-850c-7182b3f7ae3c None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:30.448478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 653/1312] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:30.453885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29243c7d-97ab-491a-9fe1-1ba8bff22fa8 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:30.454392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29243c7d-97ab-491a-9fe1-1ba8bff22fa8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:30.454473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29243c7d-97ab-491a-9fe1-1ba8bff22fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:30.547629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-29243c7d-97ab-491a-9fe1-1ba8bff22fa8 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:30.548852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-29243c7d-97ab-491a-9fe1-1ba8bff22fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:30.652066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 660/1313] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:30.706054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4de469-b878-4f1c-81a2-0dd5cbd7b262 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:30.706456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4de469-b878-4f1c-81a2-0dd5cbd7b262 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:30.706800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4de469-b878-4f1c-81a2-0dd5cbd7b262 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:30.708289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 654/1314] 10.222.0.109 () {54 vars in 970 bytes} [Thu Aug 4 22:12:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:30.716091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670de1fd-1828-4e43-9bda-39f6efacdc9f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:30.716443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670de1fd-1828-4e43-9bda-39f6efacdc9f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:30.716780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670de1fd-1828-4e43-9bda-39f6efacdc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:30.729978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:30.801587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-670de1fd-1828-4e43-9bda-39f6efacdc9f None None] MFA Rules not processed for user `5455c117e349480ba8b8c020b767bec5`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:30.802793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-670de1fd-1828-4e43-9bda-39f6efacdc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:30.883089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 661/1315] 10.222.0.109 () {58 vars in 1067 bytes} [Thu Aug 4 22:12:30 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:30.892741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4de469-b878-4f1c-81a2-0dd5cbd7b262 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:30.894356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fb4de469-b878-4f1c-81a2-0dd5cbd7b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:31.005742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4de469-b878-4f1c-81a2-0dd5cbd7b262 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:31.076604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:31.079898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b15153-28de-423d-87f1-c51d63f11b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:31.080443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15153-28de-423d-87f1-c51d63f11b85 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:31.080993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15153-28de-423d-87f1-c51d63f11b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:31.081474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15153-28de-423d-87f1-c51d63f11b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:31.083305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f5b15153-28de-423d-87f1-c51d63f11b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:31.195547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b15153-28de-423d-87f1-c51d63f11b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:31.197016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b15153-28de-423d-87f1-c51d63f11b85 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:31.215222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 655/1316] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:33.737777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29ace-7599-43fc-b7ed-35c3c9209e12 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:33.738320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29ace-7599-43fc-b7ed-35c3c9209e12 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:33.738320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29ace-7599-43fc-b7ed-35c3c9209e12 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:33.739690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 662/1317] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:33.745208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428e60-026a-49a8-912e-441b2e30327f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:33.745672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428e60-026a-49a8-912e-441b2e30327f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:33.745778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428e60-026a-49a8-912e-441b2e30327f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:33.811304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-fe428e60-026a-49a8-912e-441b2e30327f None None] MFA Rules not processed for user `411770b6026d4746a34e34ea2439c6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:33.812511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fe428e60-026a-49a8-912e-441b2e30327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:33.864280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:33.908726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 656/1318] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:33 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:33.948364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c29ace-7599-43fc-b7ed-35c3c9209e12 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:33.950060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c0c29ace-7599-43fc-b7ed-35c3c9209e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:34.055669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c29ace-7599-43fc-b7ed-35c3c9209e12 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:34.058409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:34.131900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9ac3d-7926-40e7-a144-2421458b01af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:34.133016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9ac3d-7926-40e7-a144-2421458b01af service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:34.133291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9ac3d-7926-40e7-a144-2421458b01af service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:34.133735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9ac3d-7926-40e7-a144-2421458b01af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:34.135354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0ce9ac3d-7926-40e7-a144-2421458b01af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:34.242437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9ac3d-7926-40e7-a144-2421458b01af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:34.243876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9ac3d-7926-40e7-a144-2421458b01af service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:34.262027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 663/1319] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:33 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:12:36.810025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeae643-24da-49ee-b6b9-71099f875545 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:36.810533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeae643-24da-49ee-b6b9-71099f875545 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:36.810636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeae643-24da-49ee-b6b9-71099f875545 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:36.811916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 657/1320] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:36.817014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23bae-017d-46d4-8c93-94e35b947f5b None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:36.817470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23bae-017d-46d4-8c93-94e35b947f5b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:36.817590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23bae-017d-46d4-8c93-94e35b947f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:36.859383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:36.884827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-20a23bae-017d-46d4-8c93-94e35b947f5b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:36.886076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-20a23bae-017d-46d4-8c93-94e35b947f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:36.981086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 664/1321] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:37.020265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeae643-24da-49ee-b6b9-71099f875545 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:37.021747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5aeae643-24da-49ee-b6b9-71099f875545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:37.122151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeae643-24da-49ee-b6b9-71099f875545 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:37.192100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85db0f-9804-42db-a692-4b39cd3dbbde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:37.192706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85db0f-9804-42db-a692-4b39cd3dbbde service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:37.192981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85db0f-9804-42db-a692-4b39cd3dbbde service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:37.193254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85db0f-9804-42db-a692-4b39cd3dbbde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:37.194994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1a85db0f-9804-42db-a692-4b39cd3dbbde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:37.203144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:37.303601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85db0f-9804-42db-a692-4b39cd3dbbde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:37.305491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85db0f-9804-42db-a692-4b39cd3dbbde service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:37.323464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 658/1322] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:40.021477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a3b5e-cc88-4cc7-a644-f611acf78de9 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:40.022046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a3b5e-cc88-4cc7-a644-f611acf78de9 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:40.022046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a3b5e-cc88-4cc7-a644-f611acf78de9 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:40.024042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 665/1323] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:40 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:40.029223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6d624-b11a-4b82-8ed2-1861c5c44c68 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:40.029667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6d624-b11a-4b82-8ed2-1861c5c44c68 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:40.029863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6d624-b11a-4b82-8ed2-1861c5c44c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:40.096777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-dad6d624-b11a-4b82-8ed2-1861c5c44c68 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:40.098088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-dad6d624-b11a-4b82-8ed2-1861c5c44c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:40.162436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:40.207842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 659/1324] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:40.252506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a3b5e-cc88-4cc7-a644-f611acf78de9 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:40.254205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cf0a3b5e-cc88-4cc7-a644-f611acf78de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:40.368152 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a3b5e-cc88-4cc7-a644-f611acf78de9 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:40.384797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:40.448988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d3f61-5b91-4c67-9a5c-98f113bb728f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:40.449638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d3f61-5b91-4c67-9a5c-98f113bb728f service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:40.449913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d3f61-5b91-4c67-9a5c-98f113bb728f service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:40.450179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d3f61-5b91-4c67-9a5c-98f113bb728f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:40.451781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c12d3f61-5b91-4c67-9a5c-98f113bb728f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:40.564004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d3f61-5b91-4c67-9a5c-98f113bb728f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:40.565462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d3f61-5b91-4c67-9a5c-98f113bb728f service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:40.584190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 666/1325] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:42.721297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43892-fb99-40b7-85d9-6d3452128123 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:42.721872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43892-fb99-40b7-85d9-6d3452128123 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:42.721872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43892-fb99-40b7-85d9-6d3452128123 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:42.723131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 660/1326] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:42.729289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b3922-9f8f-42ba-ae94-10e061211ad6 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:42.729934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b3922-9f8f-42ba-ae94-10e061211ad6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:42.730210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b3922-9f8f-42ba-ae94-10e061211ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:42.779123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:42.797684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-290b3922-9f8f-42ba-ae94-10e061211ad6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:42.798875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-290b3922-9f8f-42ba-ae94-10e061211ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:42.896907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 667/1327] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:42.933195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df43892-fb99-40b7-85d9-6d3452128123 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:42.934772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0df43892-fb99-40b7-85d9-6d3452128123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:43.040223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df43892-fb99-40b7-85d9-6d3452128123 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:43.116358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a316cd-4c73-49bf-a0df-e032bbbd5fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:43.117002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a316cd-4c73-49bf-a0df-e032bbbd5fac service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:43.117288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a316cd-4c73-49bf-a0df-e032bbbd5fac service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:43.117755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a316cd-4c73-49bf-a0df-e032bbbd5fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:43.119501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-96a316cd-4c73-49bf-a0df-e032bbbd5fac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:43.138865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:43.234155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a316cd-4c73-49bf-a0df-e032bbbd5fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:43.235617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a316cd-4c73-49bf-a0df-e032bbbd5fac service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:43.254264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 661/1328] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:45.386336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5420-6047-4771-ae5e-eb1e056b6a46 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:45.386879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5420-6047-4771-ae5e-eb1e056b6a46 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:45.386879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5420-6047-4771-ae5e-eb1e056b6a46 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:45.388483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 668/1329] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:45.393911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ef64c-6d5e-4d8a-b91a-0c9300390722 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:45.394330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ef64c-6d5e-4d8a-b91a-0c9300390722 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:45.394445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ef64c-6d5e-4d8a-b91a-0c9300390722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:45.462473 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-012ef64c-6d5e-4d8a-b91a-0c9300390722 None None] MFA Rules not processed for user `411770b6026d4746a34e34ea2439c6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:45.463706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-012ef64c-6d5e-4d8a-b91a-0c9300390722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:45.529593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:45.565602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 662/1330] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:45 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:45.603868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b5420-6047-4771-ae5e-eb1e056b6a46 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:45.605428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2d5b5420-6047-4771-ae5e-eb1e056b6a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:45.709221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b5420-6047-4771-ae5e-eb1e056b6a46 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:45.730432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:45.787963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fd444-d527-4740-94d5-127de2fd8e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:45.789362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fd444-d527-4740-94d5-127de2fd8e11 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:45.789694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fd444-d527-4740-94d5-127de2fd8e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:45.789965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fd444-d527-4740-94d5-127de2fd8e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:45.791674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-705fd444-d527-4740-94d5-127de2fd8e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:45.902745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fd444-d527-4740-94d5-127de2fd8e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:45.904186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fd444-d527-4740-94d5-127de2fd8e11 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:45.922064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 669/1331] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:45 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:48.201610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d79e-d2c1-47b0-82a9-553ab14f4ac1 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:48.202140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d79e-d2c1-47b0-82a9-553ab14f4ac1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:48.202140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d79e-d2c1-47b0-82a9-553ab14f4ac1 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:48.203524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 663/1332] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:48.208630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885819e0-dca9-473b-9a7d-b347da878c08 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:48.209064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885819e0-dca9-473b-9a7d-b347da878c08 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:48.209170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885819e0-dca9-473b-9a7d-b347da878c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:48.273986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:48.276829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-885819e0-dca9-473b-9a7d-b347da878c08 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:48.278671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-885819e0-dca9-473b-9a7d-b347da878c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:48.376289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 670/1333] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:48.413618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d79e-d2c1-47b0-82a9-553ab14f4ac1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:48.415153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b694d79e-d2c1-47b0-82a9-553ab14f4ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:48.517774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d79e-d2c1-47b0-82a9-553ab14f4ac1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:48.591414 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3523d-5570-4c99-a4e9-7e714d5b929f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:48.592121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3523d-5570-4c99-a4e9-7e714d5b929f service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:48.592408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3523d-5570-4c99-a4e9-7e714d5b929f service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:48.592681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3523d-5570-4c99-a4e9-7e714d5b929f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:48.594324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d9d3523d-5570-4c99-a4e9-7e714d5b929f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:48.620925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:48.703633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3523d-5570-4c99-a4e9-7e714d5b929f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:48.704999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3523d-5570-4c99-a4e9-7e714d5b929f service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:48.723191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 664/1334] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:50.799700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc4ae0-4eaa-422e-b58b-42b1625bcaa3 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:50.800132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc4ae0-4eaa-422e-b58b-42b1625bcaa3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:50.800254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc4ae0-4eaa-422e-b58b-42b1625bcaa3 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:50.801659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 671/1335] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:50.806847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d46d4-853e-4760-b025-c4d73714ed3f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:50.807276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d46d4-853e-4760-b025-c4d73714ed3f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:50.807401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d46d4-853e-4760-b025-c4d73714ed3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:50.874663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8e3d46d4-853e-4760-b025-c4d73714ed3f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:50.875915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8e3d46d4-853e-4760-b025-c4d73714ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:50.950497 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:50.973470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 665/1336] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:51.015320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc4ae0-4eaa-422e-b58b-42b1625bcaa3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:51.017066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b6bc4ae0-4eaa-422e-b58b-42b1625bcaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:51.131097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc4ae0-4eaa-422e-b58b-42b1625bcaa3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:51.159631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:51.205641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9b63a-1a97-4b8f-933a-7be6da8d55b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:51.206266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9b63a-1a97-4b8f-933a-7be6da8d55b8 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:51.206547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9b63a-1a97-4b8f-933a-7be6da8d55b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:51.206816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9b63a-1a97-4b8f-933a-7be6da8d55b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:51.208439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ffb9b63a-1a97-4b8f-933a-7be6da8d55b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:51.319326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9b63a-1a97-4b8f-933a-7be6da8d55b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:51.320893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9b63a-1a97-4b8f-933a-7be6da8d55b8 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:51.339137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 672/1337] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:53.713271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6981b6-cdb2-4cab-8cbd-cfd3b57ea275 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:53.713863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6981b6-cdb2-4cab-8cbd-cfd3b57ea275 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:53.713979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6981b6-cdb2-4cab-8cbd-cfd3b57ea275 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:53.715301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 666/1338] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:53.720671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0b208-8ff1-4fea-9e16-04ced93af0b2 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:53.721091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0b208-8ff1-4fea-9e16-04ced93af0b2 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:53.721228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0b208-8ff1-4fea-9e16-04ced93af0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:53.786531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-48e0b208-8ff1-4fea-9e16-04ced93af0b2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:53.787741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-48e0b208-8ff1-4fea-9e16-04ced93af0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:53.796598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:53.888960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 673/1339] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:53.926865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6981b6-cdb2-4cab-8cbd-cfd3b57ea275 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:53.928391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-cd6981b6-cdb2-4cab-8cbd-cfd3b57ea275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:54.033911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6981b6-cdb2-4cab-8cbd-cfd3b57ea275 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:54.108451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ebf16-f3ac-4e52-8aef-371b53985f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:54.109078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ebf16-f3ac-4e52-8aef-371b53985f46 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:54.109351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ebf16-f3ac-4e52-8aef-371b53985f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:54.109646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ebf16-f3ac-4e52-8aef-371b53985f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:54.111253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-780ebf16-f3ac-4e52-8aef-371b53985f46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:54.146329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:54.217886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ebf16-f3ac-4e52-8aef-371b53985f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:54.219317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ebf16-f3ac-4e52-8aef-371b53985f46 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:54.237223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 667/1340] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:12:58.178209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af705a-5ff8-4d92-b16f-3ee3b1f41e14 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:58.178780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af705a-5ff8-4d92-b16f-3ee3b1f41e14 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:58.178780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af705a-5ff8-4d92-b16f-3ee3b1f41e14 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:58.180122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 674/1341] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:12:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:12:58.185901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc76a9-e6b3-44b4-b025-094feec267da None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:58.186332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc76a9-e6b3-44b4-b025-094feec267da None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:58.186451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc76a9-e6b3-44b4-b025-094feec267da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:58.253699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-aabc76a9-e6b3-44b4-b025-094feec267da None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:12:58.254911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-aabc76a9-e6b3-44b4-b025-094feec267da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:58.353166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:58.365335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 668/1342] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:12:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:12:58.409071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af705a-5ff8-4d92-b16f-3ee3b1f41e14 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:12:58.410793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-00af705a-5ff8-4d92-b16f-3ee3b1f41e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:58.518214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af705a-5ff8-4d92-b16f-3ee3b1f41e14 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:12:58.557670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:12:58.593511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e97e9-a772-4c54-a34e-9496f7372b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:12:58.594133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e97e9-a772-4c54-a34e-9496f7372b5e service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:12:58.594406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e97e9-a772-4c54-a34e-9496f7372b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:12:58.594955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e97e9-a772-4c54-a34e-9496f7372b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:12:58.596578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-164e97e9-a772-4c54-a34e-9496f7372b5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:12:58.710421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e97e9-a772-4c54-a34e-9496f7372b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:12:58.712034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e97e9-a772-4c54-a34e-9496f7372b5e service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:12:58.731097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 675/1343] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:12:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:01.417055 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f3779-6b75-46f4-9684-ddfca110430d None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:01.417606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f3779-6b75-46f4-9684-ddfca110430d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:01.417606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f3779-6b75-46f4-9684-ddfca110430d None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:01.418957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 669/1344] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:13:01.424666 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa6073-9cd0-4dd2-b2ec-5e1d401c7727 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:01.425178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa6073-9cd0-4dd2-b2ec-5e1d401c7727 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:01.425293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa6073-9cd0-4dd2-b2ec-5e1d401c7727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:01.499955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-03fa6073-9cd0-4dd2-b2ec-5e1d401c7727 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:01.501225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-03fa6073-9cd0-4dd2-b2ec-5e1d401c7727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:01.520967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:01.603987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 676/1345] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:01.642161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f3779-6b75-46f4-9684-ddfca110430d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:01.643686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f1f3779-6b75-46f4-9684-ddfca110430d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:01.746010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f3779-6b75-46f4-9684-ddfca110430d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:01.817757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f1245-ae3b-4014-95ac-7eb6fd8117ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:01.818287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f1245-ae3b-4014-95ac-7eb6fd8117ba service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:01.818598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f1245-ae3b-4014-95ac-7eb6fd8117ba service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:01.818823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f1245-ae3b-4014-95ac-7eb6fd8117ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:01.820428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-376f1245-ae3b-4014-95ac-7eb6fd8117ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:01.873566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:01.943386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f1245-ae3b-4014-95ac-7eb6fd8117ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:01.944843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f1245-ae3b-4014-95ac-7eb6fd8117ba service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:01.964179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 670/1346] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:13:05.946217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba01479-5cfc-406e-bc2e-165ae7a897c3 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:05.947226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba01479-5cfc-406e-bc2e-165ae7a897c3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:05.947509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba01479-5cfc-406e-bc2e-165ae7a897c3 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:05.948876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 677/1347] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:05.954172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326ae78-02df-4613-9477-98ce45ad5fcf None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:05.954599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326ae78-02df-4613-9477-98ce45ad5fcf None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:05.954795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326ae78-02df-4613-9477-98ce45ad5fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:06.025875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b326ae78-02df-4613-9477-98ce45ad5fcf None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:06.027116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b326ae78-02df-4613-9477-98ce45ad5fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:06.128657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:06.131477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 671/1348] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:06.170604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba01479-5cfc-406e-bc2e-165ae7a897c3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:06.172186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6ba01479-5cfc-406e-bc2e-165ae7a897c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:06.283480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba01479-5cfc-406e-bc2e-165ae7a897c3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:06.353312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:06.390065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043852ca-da24-49d0-9805-cc8c33b5b8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:06.390694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043852ca-da24-49d0-9805-cc8c33b5b8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:06.390972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043852ca-da24-49d0-9805-cc8c33b5b8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:06.391240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043852ca-da24-49d0-9805-cc8c33b5b8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:06.392892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-043852ca-da24-49d0-9805-cc8c33b5b8f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:06.514856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043852ca-da24-49d0-9805-cc8c33b5b8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:06.516307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043852ca-da24-49d0-9805-cc8c33b5b8f9 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:06.536030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 678/1349] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:11.670832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b25e0-0f99-4067-8565-a1f64dc01708 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:11.671820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b25e0-0f99-4067-8565-a1f64dc01708 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:11.672238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b25e0-0f99-4067-8565-a1f64dc01708 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:11.674101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 672/1350] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:13:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:11.680106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48414d-06e4-41af-8511-a0658b48d72f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:11.680598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48414d-06e4-41af-8511-a0658b48d72f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:11.680799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48414d-06e4-41af-8511-a0658b48d72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:11.821608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2f48414d-06e4-41af-8511-a0658b48d72f None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:11.822799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2f48414d-06e4-41af-8511-a0658b48d72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:11.842794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:11.938571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 679/1351] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:13:11 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:16.115317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0628d37-faa7-4937-9cf8-8e4970a51635 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:16.115751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0628d37-faa7-4937-9cf8-8e4970a51635 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:16.115836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0628d37-faa7-4937-9cf8-8e4970a51635 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:16.118023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 673/1352] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:13:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:16.123371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186067dc-5240-41ff-bea9-bdcf5355b2e7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:16.123805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186067dc-5240-41ff-bea9-bdcf5355b2e7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:16.124105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186067dc-5240-41ff-bea9-bdcf5355b2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:16.208254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-186067dc-5240-41ff-bea9-bdcf5355b2e7 None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:16.209489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-186067dc-5240-41ff-bea9-bdcf5355b2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:16.284893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:16.323240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 680/1353] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:13:16 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:17.697090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85f156-9f2b-43dc-9da0-b953ecf88c6b None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:17.697754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85f156-9f2b-43dc-9da0-b953ecf88c6b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:17.697946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85f156-9f2b-43dc-9da0-b953ecf88c6b None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:17.699393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 674/1354] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:13:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:17.704518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703bacb-67dc-44b6-a874-d07e54a0102d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:17.704934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703bacb-67dc-44b6-a874-d07e54a0102d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:17.705054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703bacb-67dc-44b6-a874-d07e54a0102d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:17.782684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-0703bacb-67dc-44b6-a874-d07e54a0102d None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:17.784026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0703bacb-67dc-44b6-a874-d07e54a0102d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:17.894481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 681/1355] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:13:17 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:13:21.358090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6428d1-f1d4-448a-9985-a6c4000da2e8 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:21.359091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6428d1-f1d4-448a-9985-a6c4000da2e8 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:21.359376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6428d1-f1d4-448a-9985-a6c4000da2e8 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:21.360916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 675/1356] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:21.366308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171e7c2-3ce9-4b63-a408-32539c99f5d2 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:21.366776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171e7c2-3ce9-4b63-a408-32539c99f5d2 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:21.367005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171e7c2-3ce9-4b63-a408-32539c99f5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:21.418507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:21.457636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2171e7c2-3ce9-4b63-a408-32539c99f5d2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:21.458906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2171e7c2-3ce9-4b63-a408-32539c99f5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:21.566187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 682/1357] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:21.635120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45b0ad-034d-41dd-b6a9-3882f6f984cd None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:21.636161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45b0ad-034d-41dd-b6a9-3882f6f984cd None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:21.637230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45b0ad-034d-41dd-b6a9-3882f6f984cd None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:21.641328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 676/1358] 10.222.0.109 () {54 vars in 967 bytes} [Thu Aug 4 22:13:21 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:13:21.646934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d5062-6a52-4f7d-aff7-511d28136760 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:21.647216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d5062-6a52-4f7d-aff7-511d28136760 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:21.647488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d5062-6a52-4f7d-aff7-511d28136760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:21.731179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6b9d5062-6a52-4f7d-aff7-511d28136760 None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:21.732676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6b9d5062-6a52-4f7d-aff7-511d28136760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:21.802448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:21.883091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 683/1359] 10.222.0.109 () {58 vars in 1064 bytes} [Thu Aug 4 22:13:21 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:21.891249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45b0ad-034d-41dd-b6a9-3882f6f984cd None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:21.892768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ba45b0ad-034d-41dd-b6a9-3882f6f984cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:22.048771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45b0ad-034d-41dd-b6a9-3882f6f984cd None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:22.134366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:22.182966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a841d06-f28c-4edc-b23b-09e005858137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:22.183751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841d06-f28c-4edc-b23b-09e005858137 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:22.184131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841d06-f28c-4edc-b23b-09e005858137 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:22.184530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841d06-f28c-4edc-b23b-09e005858137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:22.186352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3a841d06-f28c-4edc-b23b-09e005858137 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:22.281127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a841d06-f28c-4edc-b23b-09e005858137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:22.283062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a841d06-f28c-4edc-b23b-09e005858137 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:22.303048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 677/1360] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:24.544874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780755-e9b6-4a50-b5e6-014a35f267da None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:24.545507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780755-e9b6-4a50-b5e6-014a35f267da None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:24.545507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780755-e9b6-4a50-b5e6-014a35f267da None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:24.548707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 684/1361] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:24 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:24.552104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b7f82-8ee5-422f-bf45-9cc789157d2d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:24.552545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b7f82-8ee5-422f-bf45-9cc789157d2d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:24.552648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b7f82-8ee5-422f-bf45-9cc789157d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:24.621780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-217b7f82-8ee5-422f-bf45-9cc789157d2d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:24.622991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-217b7f82-8ee5-422f-bf45-9cc789157d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:24.672838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:24.749732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 678/1362] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:24.791348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53780755-e9b6-4a50-b5e6-014a35f267da None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:24.793973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-53780755-e9b6-4a50-b5e6-014a35f267da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:24.971702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53780755-e9b6-4a50-b5e6-014a35f267da None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:24.990740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:25.096988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d30e23-df78-4164-b9fa-0475f20a6893 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:25.097934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d30e23-df78-4164-b9fa-0475f20a6893 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:25.098341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d30e23-df78-4164-b9fa-0475f20a6893 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:25.098625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d30e23-df78-4164-b9fa-0475f20a6893 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:25.100280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-45d30e23-df78-4164-b9fa-0475f20a6893 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:25.199792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d30e23-df78-4164-b9fa-0475f20a6893 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:25.201429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d30e23-df78-4164-b9fa-0475f20a6893 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:25.220711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 685/1363] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:13:27.608158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043310fe-799f-4b3f-a271-8cfc2b2f0dea None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:27.608689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043310fe-799f-4b3f-a271-8cfc2b2f0dea None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:27.608689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043310fe-799f-4b3f-a271-8cfc2b2f0dea None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:27.610210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 679/1364] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:13:27.619004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885f72-df32-41ff-992d-acc102b9be69 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:27.619552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885f72-df32-41ff-992d-acc102b9be69 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:27.619552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885f72-df32-41ff-992d-acc102b9be69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:27.640123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:27.704706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ed885f72-df32-41ff-992d-acc102b9be69 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:27.705990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ed885f72-df32-41ff-992d-acc102b9be69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:27.811403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 686/1365] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:27.863315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ace9dc-3515-475c-bac1-dd3fb6624d63 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:27.864070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ace9dc-3515-475c-bac1-dd3fb6624d63 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:27.864579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ace9dc-3515-475c-bac1-dd3fb6624d63 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:27.866087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 680/1366] 10.222.0.109 () {54 vars in 967 bytes} [Thu Aug 4 22:13:27 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:27.871608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc59de-ecc8-4559-ac62-fbf571eee1f2 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:27.871899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc59de-ecc8-4559-ac62-fbf571eee1f2 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:27.872177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc59de-ecc8-4559-ac62-fbf571eee1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:27.956375 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f9bc59de-ecc8-4559-ac62-fbf571eee1f2 None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:27.957618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f9bc59de-ecc8-4559-ac62-fbf571eee1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:27.997148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:28.087989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 687/1367] 10.222.0.109 () {58 vars in 1064 bytes} [Thu Aug 4 22:13:27 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:28.095777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ace9dc-3515-475c-bac1-dd3fb6624d63 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:28.097720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a3ace9dc-3515-475c-bac1-dd3fb6624d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:28.294870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ace9dc-3515-475c-bac1-dd3fb6624d63 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:28.315949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:28.416159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bab6c-0d3c-43de-8294-b07426e4a34a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:28.416791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bab6c-0d3c-43de-8294-b07426e4a34a service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:28.417070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bab6c-0d3c-43de-8294-b07426e4a34a service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:28.417335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bab6c-0d3c-43de-8294-b07426e4a34a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:28.419009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-cf6bab6c-0d3c-43de-8294-b07426e4a34a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:28.512978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bab6c-0d3c-43de-8294-b07426e4a34a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:28.514620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bab6c-0d3c-43de-8294-b07426e4a34a service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:28.533596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 681/1368] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:29.479458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff592-d2de-4904-a174-2fae8733665f None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:29.479988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff592-d2de-4904-a174-2fae8733665f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:29.479988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ff592-d2de-4904-a174-2fae8733665f None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:29.481399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 688/1369] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:13:29.486339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b4d53-629a-41dd-9b37-66996df7f0dc None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:29.486745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b4d53-629a-41dd-9b37-66996df7f0dc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:29.486925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b4d53-629a-41dd-9b37-66996df7f0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:29.521467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:29.561664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-398b4d53-629a-41dd-9b37-66996df7f0dc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:29.563004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-398b4d53-629a-41dd-9b37-66996df7f0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:29.665465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 682/1370] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:30.004879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8eaf5-22b6-4029-9957-81c42ebf552f None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:30.005162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8eaf5-22b6-4029-9957-81c42ebf552f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:30.005464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8eaf5-22b6-4029-9957-81c42ebf552f None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:30.007062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 689/1371] 10.222.0.109 () {54 vars in 969 bytes} [Thu Aug 4 22:13:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:30.011993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b882-4007-4301-b30e-507dc747c6bc None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:30.012282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b882-4007-4301-b30e-507dc747c6bc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:30.012812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b882-4007-4301-b30e-507dc747c6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:30.094887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a5c5b882-4007-4301-b30e-507dc747c6bc None None] MFA Rules not processed for user `1d0422cd82114045a9fff7df29bc3906`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:30.096377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a5c5b882-4007-4301-b30e-507dc747c6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:30.150176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:30.182060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 683/1372] 10.222.0.109 () {58 vars in 1066 bytes} [Thu Aug 4 22:13:30 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:30.189757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8eaf5-22b6-4029-9957-81c42ebf552f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:30.191373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7fc8eaf5-22b6-4029-9957-81c42ebf552f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:30.296581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8eaf5-22b6-4029-9957-81c42ebf552f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:30.340334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:30.376046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2817496-baef-4b1f-bc35-dcd452406e1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:30.376798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817496-baef-4b1f-bc35-dcd452406e1a service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:30.377175 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817496-baef-4b1f-bc35-dcd452406e1a service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:30.377574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817496-baef-4b1f-bc35-dcd452406e1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:30.379282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e2817496-baef-4b1f-bc35-dcd452406e1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:30.489303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2817496-baef-4b1f-bc35-dcd452406e1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:30.490779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2817496-baef-4b1f-bc35-dcd452406e1a service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:30.511157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 690/1373] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:30.672192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793857d-cfc4-477f-b22f-9eed3fa5d1d0 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:30.672192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793857d-cfc4-477f-b22f-9eed3fa5d1d0 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:30.672727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793857d-cfc4-477f-b22f-9eed3fa5d1d0 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:30.674605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 684/1374] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:30.680980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba9592-bc9d-440b-abda-eb5da65f8f76 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:30.681268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba9592-bc9d-440b-abda-eb5da65f8f76 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:30.681649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba9592-bc9d-440b-abda-eb5da65f8f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:30.770810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-0aba9592-bc9d-440b-abda-eb5da65f8f76 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:30.772050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0aba9592-bc9d-440b-abda-eb5da65f8f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:30.797806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:30.883488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 691/1375] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:30.931717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793857d-cfc4-477f-b22f-9eed3fa5d1d0 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:30.933280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2793857d-cfc4-477f-b22f-9eed3fa5d1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:31.185049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793857d-cfc4-477f-b22f-9eed3fa5d1d0 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:31.245905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:31.315018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c57b62-85df-4420-aef6-03f22e1704e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:31.315655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c57b62-85df-4420-aef6-03f22e1704e5 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:31.316050 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c57b62-85df-4420-aef6-03f22e1704e5 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:31.316325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c57b62-85df-4420-aef6-03f22e1704e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:31.317987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b7c57b62-85df-4420-aef6-03f22e1704e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:31.415694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c57b62-85df-4420-aef6-03f22e1704e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:31.417220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c57b62-85df-4420-aef6-03f22e1704e5 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:31.439400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 685/1376] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:33.508028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ecac-2eca-4d2c-b154-7e92b8702498 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:33.508558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ecac-2eca-4d2c-b154-7e92b8702498 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:33.508676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ecac-2eca-4d2c-b154-7e92b8702498 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:33.510064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 692/1377] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:13:33.515591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a01bbc-648c-430a-b32a-8448e1e2954b None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:33.516034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a01bbc-648c-430a-b32a-8448e1e2954b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:33.516141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a01bbc-648c-430a-b32a-8448e1e2954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:33.594454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f1a01bbc-648c-430a-b32a-8448e1e2954b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:33.597306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f1a01bbc-648c-430a-b32a-8448e1e2954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:33.612550 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:33.711350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 686/1378] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:33.755502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3ecac-2eca-4d2c-b154-7e92b8702498 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:33.758109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-40e3ecac-2eca-4d2c-b154-7e92b8702498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:33.910307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3ecac-2eca-4d2c-b154-7e92b8702498 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:33.914439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:34.063032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12c60c-1e7f-4162-97a3-5bb3b79c07ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:34.064052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12c60c-1e7f-4162-97a3-5bb3b79c07ce service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:34.064548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12c60c-1e7f-4162-97a3-5bb3b79c07ce service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:34.065172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12c60c-1e7f-4162-97a3-5bb3b79c07ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:34.067445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2f12c60c-1e7f-4162-97a3-5bb3b79c07ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:34.185811 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12c60c-1e7f-4162-97a3-5bb3b79c07ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:34.187432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12c60c-1e7f-4162-97a3-5bb3b79c07ce service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:34.206354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 693/1379] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:36.288789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef013344-ce72-45c6-b2a4-fe654486fb4b None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:36.289353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef013344-ce72-45c6-b2a4-fe654486fb4b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:36.289353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef013344-ce72-45c6-b2a4-fe654486fb4b None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:36.290911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 687/1380] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:36.295951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f164e-8282-4ccf-85a7-8f2f35bf38cb None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:36.296408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f164e-8282-4ccf-85a7-8f2f35bf38cb None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:36.296508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f164e-8282-4ccf-85a7-8f2f35bf38cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:36.308215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:36.369000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-809f164e-8282-4ccf-85a7-8f2f35bf38cb None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:36.370150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-809f164e-8282-4ccf-85a7-8f2f35bf38cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:36.480409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 694/1381] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:36.525007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef013344-ce72-45c6-b2a4-fe654486fb4b None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:36.526570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ef013344-ce72-45c6-b2a4-fe654486fb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:36.669090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:36.680813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef013344-ce72-45c6-b2a4-fe654486fb4b None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:36.803266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdff6fb-b5fe-4bfb-85a4-8e6574652b19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:36.803955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdff6fb-b5fe-4bfb-85a4-8e6574652b19 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:36.804278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdff6fb-b5fe-4bfb-85a4-8e6574652b19 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:36.804555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdff6fb-b5fe-4bfb-85a4-8e6574652b19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:36.806207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5bdff6fb-b5fe-4bfb-85a4-8e6574652b19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:36.912608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdff6fb-b5fe-4bfb-85a4-8e6574652b19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:36.914530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdff6fb-b5fe-4bfb-85a4-8e6574652b19 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:36.927677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:36.938910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 688/1382] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:38.989379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276be0d-8c31-4d06-984e-4ae3811ffdc8 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:38.989971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276be0d-8c31-4d06-984e-4ae3811ffdc8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:38.989971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276be0d-8c31-4d06-984e-4ae3811ffdc8 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:38.991925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 695/1383] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:38.997661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23507fc-5752-4319-a131-2f52ac8b27a4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:38.998416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23507fc-5752-4319-a131-2f52ac8b27a4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:38.998695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23507fc-5752-4319-a131-2f52ac8b27a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:39.067456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f23507fc-5752-4319-a131-2f52ac8b27a4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:39.068599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f23507fc-5752-4319-a131-2f52ac8b27a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:39.224301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 689/1384] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:13:39.272321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276be0d-8c31-4d06-984e-4ae3811ffdc8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:39.274627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9276be0d-8c31-4d06-984e-4ae3811ffdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:39.364153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:39.458352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276be0d-8c31-4d06-984e-4ae3811ffdc8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:39.607404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f188b9-dede-431d-9836-f5f9032cbb75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:39.608035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f188b9-dede-431d-9836-f5f9032cbb75 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:39.608315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f188b9-dede-431d-9836-f5f9032cbb75 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:39.608580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f188b9-dede-431d-9836-f5f9032cbb75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:39.610253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c6f188b9-dede-431d-9836-f5f9032cbb75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:39.660792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:39.703259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f188b9-dede-431d-9836-f5f9032cbb75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:39.704975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f188b9-dede-431d-9836-f5f9032cbb75 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:39.724282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 696/1385] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:42.280860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f560094-19e3-4521-955e-d52a58895058 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:42.281391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f560094-19e3-4521-955e-d52a58895058 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:42.281888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f560094-19e3-4521-955e-d52a58895058 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:42.284344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 690/1386] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:42 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:42.290323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3163c-1691-46c9-8371-c137cf319470 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:42.290742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3163c-1691-46c9-8371-c137cf319470 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:42.290875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3163c-1691-46c9-8371-c137cf319470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:42.360869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3bd3163c-1691-46c9-8371-c137cf319470 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:42.362104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3bd3163c-1691-46c9-8371-c137cf319470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:42.474438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 697/1387] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:42.518471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f560094-19e3-4521-955e-d52a58895058 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:42.520029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f560094-19e3-4521-955e-d52a58895058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:42.596160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:42.671077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f560094-19e3-4521-955e-d52a58895058 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:42.817934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c20c68-fa82-4a49-8d7d-b2f7149741cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:42.819062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20c68-fa82-4a49-8d7d-b2f7149741cd service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:42.819062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20c68-fa82-4a49-8d7d-b2f7149741cd service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:42.819730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20c68-fa82-4a49-8d7d-b2f7149741cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:42.821337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b0c20c68-fa82-4a49-8d7d-b2f7149741cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:42.875401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:42.955268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c20c68-fa82-4a49-8d7d-b2f7149741cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:42.958223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c20c68-fa82-4a49-8d7d-b2f7149741cd service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:42.988995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 691/1388] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:13:45.014263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1af77-ba55-4fa6-b292-91e955ca70b5 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:45.014835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1af77-ba55-4fa6-b292-91e955ca70b5 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:45.014835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1af77-ba55-4fa6-b292-91e955ca70b5 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:45.016066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 698/1389] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:45.021158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a903f-52ff-40fa-828e-e0a17171a8db None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:45.021637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a903f-52ff-40fa-828e-e0a17171a8db None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:45.021734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a903f-52ff-40fa-828e-e0a17171a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:45.091683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-614a903f-52ff-40fa-828e-e0a17171a8db None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:45.092926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-614a903f-52ff-40fa-828e-e0a17171a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:45.195568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 692/1390] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:45.240372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1af77-ba55-4fa6-b292-91e955ca70b5 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:45.242178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-afe1af77-ba55-4fa6-b292-91e955ca70b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:45.274580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:45.393472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1af77-ba55-4fa6-b292-91e955ca70b5 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:45.498866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:45.512700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c72ac1-471a-4390-b256-b271bea812e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:45.513319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c72ac1-471a-4390-b256-b271bea812e8 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:45.513692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c72ac1-471a-4390-b256-b271bea812e8 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:45.513961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c72ac1-471a-4390-b256-b271bea812e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:45.515573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-18c72ac1-471a-4390-b256-b271bea812e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:45.606831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c72ac1-471a-4390-b256-b271bea812e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:45.608440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c72ac1-471a-4390-b256-b271bea812e8 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:45.626282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 699/1391] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:45.771639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42176e-2ed7-4081-8ae1-9aef86c5255a None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:45.771956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42176e-2ed7-4081-8ae1-9aef86c5255a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:45.772231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42176e-2ed7-4081-8ae1-9aef86c5255a None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:45.774299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 693/1392] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:45.780736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b19103-0a73-4677-8394-bd69496aafa1 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:45.781021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b19103-0a73-4677-8394-bd69496aafa1 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:45.781312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b19103-0a73-4677-8394-bd69496aafa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:45.855328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f8b19103-0a73-4677-8394-bd69496aafa1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:45.856555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f8b19103-0a73-4677-8394-bd69496aafa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:45.915076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:45.954028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 700/1393] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:46.011721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a758a-a20c-4079-9893-40f807cd0ae3 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:46.012006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a758a-a20c-4079-9893-40f807cd0ae3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:46.012545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a758a-a20c-4079-9893-40f807cd0ae3 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:46.013988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 694/1394] 10.222.0.109 () {54 vars in 971 bytes} [Thu Aug 4 22:13:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:46.019261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3643-10b3-4ea1-9a5d-6c9e71a1d23e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:46.019722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3643-10b3-4ea1-9a5d-6c9e71a1d23e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:46.019834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3643-10b3-4ea1-9a5d-6c9e71a1d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:46.095898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e68f3643-10b3-4ea1-9a5d-6c9e71a1d23e None None] MFA Rules not processed for user `33183e28de8f487f82f92f0151e7c73c`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:46.097089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e68f3643-10b3-4ea1-9a5d-6c9e71a1d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:46.176150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 701/1395] 10.222.0.109 () {58 vars in 1068 bytes} [Thu Aug 4 22:13:46 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:46.184340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a758a-a20c-4079-9893-40f807cd0ae3 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:46.185968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f1a758a-a20c-4079-9893-40f807cd0ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:46.199978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:46.290766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a758a-a20c-4079-9893-40f807cd0ae3 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:46.364109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff6e31-e462-4b47-a2d6-21a9626fe6c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33183e28de8f487f82f92f0151e7c73c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:46.364732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff6e31-e462-4b47-a2d6-21a9626fe6c0 service glance] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:46.365011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff6e31-e462-4b47-a2d6-21a9626fe6c0 service glance] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:46.365279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff6e31-e462-4b47-a2d6-21a9626fe6c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:46.367614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-56ff6e31-e462-4b47-a2d6-21a9626fe6c0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:46.440412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:46.482165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff6e31-e462-4b47-a2d6-21a9626fe6c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:46.483640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff6e31-e462-4b47-a2d6-21a9626fe6c0 service glance] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:46.502212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 695/1396] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:47.263752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015fb69-784e-4414-a46a-f7f23314d4ba None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:47.264275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015fb69-784e-4414-a46a-f7f23314d4ba None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:47.264379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015fb69-784e-4414-a46a-f7f23314d4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:47.331670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4015fb69-784e-4414-a46a-f7f23314d4ba None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:47.332914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4015fb69-784e-4414-a46a-f7f23314d4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:47.342910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:47.423581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 702/1397] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:13:47 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:47.468165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a1315-cf1b-4334-a033-09c3f6e58b83 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:47.468712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a1315-cf1b-4334-a033-09c3f6e58b83 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:47.468820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a1315-cf1b-4334-a033-09c3f6e58b83 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:47.470315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 696/1398] 10.222.0.109 () {54 vars in 948 bytes} [Thu Aug 4 22:13:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:47.477654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9f979-8162-4c72-b765-99cd2d9fca05 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:47.477962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9f979-8162-4c72-b765-99cd2d9fca05 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:47.478280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9f979-8162-4c72-b765-99cd2d9fca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:47.557190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-bfe9f979-8162-4c72-b765-99cd2d9fca05 None None] MFA Rules not processed for user `848151a99fb74f6b8f8210cfc49afeff`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:47.558422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bfe9f979-8162-4c72-b765-99cd2d9fca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:47.646072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:47.669353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 703/1399] 10.222.0.109 () {58 vars in 1045 bytes} [Thu Aug 4 22:13:47 2022] POST /identity/v3/auth/tokens => generated 3314 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:47.679838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a1315-cf1b-4334-a033-09c3f6e58b83 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:47.681516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1a7a1315-cf1b-4334-a033-09c3f6e58b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:47.725778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08871ef9-82ec-4f35-91c0-29a1d3449fcd None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:47.726223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08871ef9-82ec-4f35-91c0-29a1d3449fcd None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:47.726504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08871ef9-82ec-4f35-91c0-29a1d3449fcd None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:47.728066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 704/1400] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:13:47.732985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f0632-00b2-43ba-bce6-18368486d540 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:47.733457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f0632-00b2-43ba-bce6-18368486d540 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:47.733733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f0632-00b2-43ba-bce6-18368486d540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:47.802249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7a5f0632-00b2-43ba-bce6-18368486d540 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:47.804014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7a5f0632-00b2-43ba-bce6-18368486d540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:47.822650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a1315-cf1b-4334-a033-09c3f6e58b83 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:47.860082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:47.905756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 705/1401] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:13:47.930234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4bae2-d09a-4c97-b948-0f4860819576 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:47.930777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bae2-d09a-4c97-b948-0f4860819576 service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:47.931058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bae2-d09a-4c97-b948-0f4860819576 service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:47.931332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bae2-d09a-4c97-b948-0f4860819576 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:47.933014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-97d4bae2-d09a-4c97-b948-0f4860819576 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:47.950504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f0632-00b2-43ba-bce6-18368486d540 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:47.951698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7a5f0632-00b2-43ba-bce6-18368486d540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:48.018032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:48.019852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4bae2-d09a-4c97-b948-0f4860819576 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:48.021643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4bae2-d09a-4c97-b948-0f4860819576 service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:48.030559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 697/1402] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:13:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 22:13:48.074600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f0632-00b2-43ba-bce6-18368486d540 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:48.220287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e944-fd0a-475d-a27a-1634345e2e10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:48.220811 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e944-fd0a-475d-a27a-1634345e2e10 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:48.221084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e944-fd0a-475d-a27a-1634345e2e10 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:48.221348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e944-fd0a-475d-a27a-1634345e2e10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:48.224790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-da47e944-fd0a-475d-a27a-1634345e2e10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:48.294739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:48.331387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47e944-fd0a-475d-a27a-1634345e2e10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:48.333009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47e944-fd0a-475d-a27a-1634345e2e10 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:48.356782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 706/1403] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:13:50.431215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59463a-ab3c-489c-af6d-5081c34d3723 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:50.431739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59463a-ab3c-489c-af6d-5081c34d3723 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:50.431739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59463a-ab3c-489c-af6d-5081c34d3723 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:50.437081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 698/1404] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:50 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:50.438741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f0319-2e30-4afc-a818-9483a44fd58c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:50.439179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f0319-2e30-4afc-a818-9483a44fd58c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:50.439274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f0319-2e30-4afc-a818-9483a44fd58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:50.515133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2f5f0319-2e30-4afc-a818-9483a44fd58c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:50.516577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2f5f0319-2e30-4afc-a818-9483a44fd58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:50.628702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 707/1405] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:13:50.675029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59463a-ab3c-489c-af6d-5081c34d3723 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:50.676613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5b59463a-ab3c-489c-af6d-5081c34d3723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:50.759442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:50.864845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59463a-ab3c-489c-af6d-5081c34d3723 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:51.075587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea7008-858d-4316-b242-b6f9e48d3ff4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:51.076383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea7008-858d-4316-b242-b6f9e48d3ff4 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:51.076700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea7008-858d-4316-b242-b6f9e48d3ff4 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:51.076987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea7008-858d-4316-b242-b6f9e48d3ff4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:51.078976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f6ea7008-858d-4316-b242-b6f9e48d3ff4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:51.137645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:51.217334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea7008-858d-4316-b242-b6f9e48d3ff4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:51.219751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea7008-858d-4316-b242-b6f9e48d3ff4 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:51.247117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 699/1406] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:13:53.427739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904847f8-00a0-49ff-9635-ebf5a0f718ff None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:53.428280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904847f8-00a0-49ff-9635-ebf5a0f718ff None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:53.428280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904847f8-00a0-49ff-9635-ebf5a0f718ff None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:53.430028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 708/1407] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:13:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:13:53.434898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4370e5-ab4d-4983-944e-13a2cfe8c73b None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:53.435358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4370e5-ab4d-4983-944e-13a2cfe8c73b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:53.435494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4370e5-ab4d-4983-944e-13a2cfe8c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:53.508830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9f4370e5-ab4d-4983-944e-13a2cfe8c73b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:13:53.510122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f4370e5-ab4d-4983-944e-13a2cfe8c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:53.627870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 700/1408] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:13:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:13:53.674414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904847f8-00a0-49ff-9635-ebf5a0f718ff None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:13:53.676120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-904847f8-00a0-49ff-9635-ebf5a0f718ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:53.719046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:53.836186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904847f8-00a0-49ff-9635-ebf5a0f718ff None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:13:53.963184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:13:53.969984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8230bb8-3d92-4b83-b758-17f055b4eb98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:13:53.970575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8230bb8-3d92-4b83-b758-17f055b4eb98 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:13:53.970845 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8230bb8-3d92-4b83-b758-17f055b4eb98 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:13:53.971112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8230bb8-3d92-4b83-b758-17f055b4eb98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:13:53.972729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b8230bb8-3d92-4b83-b758-17f055b4eb98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:13:54.073322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8230bb8-3d92-4b83-b758-17f055b4eb98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:13:54.075062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8230bb8-3d92-4b83-b758-17f055b4eb98 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:13:54.096797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 709/1409] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:13:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:03.626062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:04.094407 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=47 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:10.217480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730e87c-38c9-4c3c-b6a3-1498a769f6f3 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:10.218023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730e87c-38c9-4c3c-b6a3-1498a769f6f3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:10.218023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730e87c-38c9-4c3c-b6a3-1498a769f6f3 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:10.219386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 701/1410] 10.222.0.109 () {54 vars in 923 bytes} [Thu Aug 4 22:14:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:10.224545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5343bc2-1087-4031-9c4e-11fb359eab92 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:10.225061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5343bc2-1087-4031-9c4e-11fb359eab92 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:10.225164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5343bc2-1087-4031-9c4e-11fb359eab92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:10.300814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f5343bc2-1087-4031-9c4e-11fb359eab92 None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:10.302048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f5343bc2-1087-4031-9c4e-11fb359eab92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:10.409498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 710/1411] 10.222.0.109 () {58 vars in 1020 bytes} [Thu Aug 4 22:14:10 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:10.423796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc3599-0d7f-475e-abad-2f963526e3b2 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:10.424087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc3599-0d7f-475e-abad-2f963526e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:10.424424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc3599-0d7f-475e-abad-2f963526e3b2 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:10.425815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 702/1412] 10.222.0.109 () {54 vars in 923 bytes} [Thu Aug 4 22:14:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:10.431208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2ef3-6ebd-42a4-834e-b7008bf17027 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:10.431497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2ef3-6ebd-42a4-834e-b7008bf17027 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:10.431778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2ef3-6ebd-42a4-834e-b7008bf17027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:10.505242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:10.508578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-01bb2ef3-6ebd-42a4-834e-b7008bf17027 None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:10.510397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-01bb2ef3-6ebd-42a4-834e-b7008bf17027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:10.623507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 711/1413] 10.222.0.109 () {58 vars in 1020 bytes} [Thu Aug 4 22:14:10 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:14:13.075926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed86ab-d3ae-435b-9e9d-c4848c5f0f48 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:13.076451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed86ab-d3ae-435b-9e9d-c4848c5f0f48 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:13.076451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed86ab-d3ae-435b-9e9d-c4848c5f0f48 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:13.077780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:13.082907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eef6c-1c71-43ab-ab16-1bc418b117eb None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:13.083355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eef6c-1c71-43ab-ab16-1bc418b117eb None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:13.083461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eef6c-1c71-43ab-ab16-1bc418b117eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:13.151089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e91eef6c-1c71-43ab-ab16-1bc418b117eb None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:13.152350 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e91eef6c-1c71-43ab-ab16-1bc418b117eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:13.216189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:13.260188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:14:13.305314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed86ab-d3ae-435b-9e9d-c4848c5f0f48 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:13.306865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-feed86ab-d3ae-435b-9e9d-c4848c5f0f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:13.450240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed86ab-d3ae-435b-9e9d-c4848c5f0f48 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:13.525740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:13.565813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69364886-a40b-4529-86f2-1f7d3dd8feaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:13.566437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69364886-a40b-4529-86f2-1f7d3dd8feaa service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:13.566717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69364886-a40b-4529-86f2-1f7d3dd8feaa service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:13.566992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69364886-a40b-4529-86f2-1f7d3dd8feaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:13.569204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-69364886-a40b-4529-86f2-1f7d3dd8feaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:13.657718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69364886-a40b-4529-86f2-1f7d3dd8feaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:13.659350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69364886-a40b-4529-86f2-1f7d3dd8feaa service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:13.676991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 704/1416] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:14.566682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04f9f-2376-4b05-bd6c-6e93095d2105 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:14.567198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04f9f-2376-4b05-bd6c-6e93095d2105 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:14.567198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04f9f-2376-4b05-bd6c-6e93095d2105 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:14.568481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:14:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:14.574045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be814-352e-455f-85c1-9fe7c4cb36dc None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:14.574445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be814-352e-455f-85c1-9fe7c4cb36dc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:14.574597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be814-352e-455f-85c1-9fe7c4cb36dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:14.650901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2b0be814-352e-455f-85c1-9fe7c4cb36dc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:14.652103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2b0be814-352e-455f-85c1-9fe7c4cb36dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:14.679170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:14.747538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:14:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:14.901660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc04f9f-2376-4b05-bd6c-6e93095d2105 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:14.903222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bdc04f9f-2376-4b05-bd6c-6e93095d2105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:15.050158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc04f9f-2376-4b05-bd6c-6e93095d2105 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:15.090115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:15.167110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e15a3e-5847-47a8-843d-342d7516891a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:15.167743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e15a3e-5847-47a8-843d-342d7516891a service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:15.168022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e15a3e-5847-47a8-843d-342d7516891a service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:15.168337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e15a3e-5847-47a8-843d-342d7516891a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:15.170077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-48e15a3e-5847-47a8-843d-342d7516891a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:15.269497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e15a3e-5847-47a8-843d-342d7516891a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:15.271091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e15a3e-5847-47a8-843d-342d7516891a service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:15.289807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 714/1419] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:16.176994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ae04-e1c3-4bf0-94fc-706a48079d25 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:16.177597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ae04-e1c3-4bf0-94fc-706a48079d25 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:16.177597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ae04-e1c3-4bf0-94fc-706a48079d25 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:16.179434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:14:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:16.184592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698199-96b5-4052-9d0a-d394117bb1f0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:16.185005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698199-96b5-4052-9d0a-d394117bb1f0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:16.185133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1698199-96b5-4052-9d0a-d394117bb1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:16.239511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:16.262877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f1698199-96b5-4052-9d0a-d394117bb1f0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:16.264088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f1698199-96b5-4052-9d0a-d394117bb1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:16.359074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:14:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:16.510689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ae04-e1c3-4bf0-94fc-706a48079d25 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:16.512501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c8e2ae04-e1c3-4bf0-94fc-706a48079d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:16.659118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ae04-e1c3-4bf0-94fc-706a48079d25 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:16.671457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:16.773831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec747e-82db-40ed-8837-35811fd0b3ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:16.774451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec747e-82db-40ed-8837-35811fd0b3ba service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:16.774731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec747e-82db-40ed-8837-35811fd0b3ba service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:16.775006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec747e-82db-40ed-8837-35811fd0b3ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:16.776643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-27ec747e-82db-40ed-8837-35811fd0b3ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:16.864537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec747e-82db-40ed-8837-35811fd0b3ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:16.866699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec747e-82db-40ed-8837-35811fd0b3ba service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:16.884883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 707/1422] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:17.754952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7fe6-1c46-4b53-9f53-89ea423c7506 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:17.755529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7fe6-1c46-4b53-9f53-89ea423c7506 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:17.755529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7fe6-1c46-4b53-9f53-89ea423c7506 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:17.757129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:14:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:17.762466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5c85-1d6f-4b42-9f81-2a9a4728d941 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:17.762903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5c85-1d6f-4b42-9f81-2a9a4728d941 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:17.763011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5c85-1d6f-4b42-9f81-2a9a4728d941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:17.782850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:17.838965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b30d5c85-1d6f-4b42-9f81-2a9a4728d941 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:17.840197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b30d5c85-1d6f-4b42-9f81-2a9a4728d941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:17.936654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:14:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:18.086700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a7fe6-1c46-4b53-9f53-89ea423c7506 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:18.088248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bd4a7fe6-1c46-4b53-9f53-89ea423c7506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:18.208105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:18.245954 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a7fe6-1c46-4b53-9f53-89ea423c7506 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:18.356251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a981f-5eba-4bf0-a9f3-a6affa47069b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:18.356787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a981f-5eba-4bf0-a9f3-a6affa47069b service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:18.357062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a981f-5eba-4bf0-a9f3-a6affa47069b service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:18.357332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a981f-5eba-4bf0-a9f3-a6affa47069b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:18.358978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9b6a981f-5eba-4bf0-a9f3-a6affa47069b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:18.433400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:18.447436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a981f-5eba-4bf0-a9f3-a6affa47069b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:18.449032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a981f-5eba-4bf0-a9f3-a6affa47069b service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:18.466548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 717/1425] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:19.335208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba277c3-4035-4973-8237-f76a11d479c4 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:19.335787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba277c3-4035-4973-8237-f76a11d479c4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:19.335787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba277c3-4035-4973-8237-f76a11d479c4 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:19.337184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:14:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:19.342111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276717d-c7dc-41c0-8932-855c13b4ff63 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:19.342542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276717d-c7dc-41c0-8932-855c13b4ff63 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:19.342691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276717d-c7dc-41c0-8932-855c13b4ff63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:19.419392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6276717d-c7dc-41c0-8932-855c13b4ff63 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:19.420518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6276717d-c7dc-41c0-8932-855c13b4ff63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:19.517764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:14:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:19.669900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba277c3-4035-4973-8237-f76a11d479c4 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:19.671359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0ba277c3-4035-4973-8237-f76a11d479c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:19.756062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:19.817190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba277c3-4035-4973-8237-f76a11d479c4 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:19.928149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca3f54-8908-42d5-85b8-42082338f912 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:19.928695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3f54-8908-42d5-85b8-42082338f912 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:19.928977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3f54-8908-42d5-85b8-42082338f912 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:19.929249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3f54-8908-42d5-85b8-42082338f912 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:19.930931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6dca3f54-8908-42d5-85b8-42082338f912 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:19.978876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:20.019526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca3f54-8908-42d5-85b8-42082338f912 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:20.021136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca3f54-8908-42d5-85b8-42082338f912 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:20.040160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 710/1428] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:20.846512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919576f-18a0-44a5-b6b9-597dcef0e61c None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:20.846936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919576f-18a0-44a5-b6b9-597dcef0e61c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:20.847053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919576f-18a0-44a5-b6b9-597dcef0e61c None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:20.848441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:14:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:14:20.853430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2a14-6ef2-4c26-9609-953dd750c83e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:20.853852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2a14-6ef2-4c26-9609-953dd750c83e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:20.854014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2a14-6ef2-4c26-9609-953dd750c83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:20.929770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-314d2a14-6ef2-4c26-9609-953dd750c83e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:20.930969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-314d2a14-6ef2-4c26-9609-953dd750c83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:21.026032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:14:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:21.177434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919576f-18a0-44a5-b6b9-597dcef0e61c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:21.180159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0919576f-18a0-44a5-b6b9-597dcef0e61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:21.229252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:21.339046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919576f-18a0-44a5-b6b9-597dcef0e61c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:21.451540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb7814-471a-4b24-9976-9ad8cd175167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:21.452070 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb7814-471a-4b24-9976-9ad8cd175167 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:21.452348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb7814-471a-4b24-9976-9ad8cd175167 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:21.452613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb7814-471a-4b24-9976-9ad8cd175167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:21.454252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5dbb7814-471a-4b24-9976-9ad8cd175167 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:21.461517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:21.543269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb7814-471a-4b24-9976-9ad8cd175167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:21.544857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb7814-471a-4b24-9976-9ad8cd175167 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:21.562424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 720/1431] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:22.372509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fe5a-d101-48c4-a1f7-05e529ec3603 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:22.373092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fe5a-d101-48c4-a1f7-05e529ec3603 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:22.373092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fe5a-d101-48c4-a1f7-05e529ec3603 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:22.374390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:14:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:22.379262 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2b636-9ace-4594-94ad-159b181e69a4 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:22.379702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2b636-9ace-4594-94ad-159b181e69a4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:22.379820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2b636-9ace-4594-94ad-159b181e69a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:22.464128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-43f2b636-9ace-4594-94ad-159b181e69a4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:22.465344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-43f2b636-9ace-4594-94ad-159b181e69a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:22.531455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:22.561344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:14:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:22.714256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734fe5a-d101-48c4-a1f7-05e529ec3603 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:22.715802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1734fe5a-d101-48c4-a1f7-05e529ec3603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:22.733754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:22.866525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734fe5a-d101-48c4-a1f7-05e529ec3603 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:22.956702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:22.984176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d58a7-4e31-4382-9055-a7cf49331a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:22.984792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d58a7-4e31-4382-9055-a7cf49331a1c service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:22.985069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d58a7-4e31-4382-9055-a7cf49331a1c service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:22.985341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d58a7-4e31-4382-9055-a7cf49331a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:22.987049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d68d58a7-4e31-4382-9055-a7cf49331a1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:23.075522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d58a7-4e31-4382-9055-a7cf49331a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:23.077198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d58a7-4e31-4382-9055-a7cf49331a1c service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:23.095652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 713/1434] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:23.915610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5864cea-2730-4299-b13a-4a3ce778dbfa None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:23.916196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5864cea-2730-4299-b13a-4a3ce778dbfa None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:23.916320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5864cea-2730-4299-b13a-4a3ce778dbfa None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:23.917761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:14:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:23.923055 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f205-8069-4074-a818-9450fd55d4c4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:23.923509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f205-8069-4074-a818-9450fd55d4c4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:23.923623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f205-8069-4074-a818-9450fd55d4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:23.996837 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-4024f205-8069-4074-a818-9450fd55d4c4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:23.998048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4024f205-8069-4074-a818-9450fd55d4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:24.038581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:24.094848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:14:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:24.251366 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5864cea-2730-4299-b13a-4a3ce778dbfa None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:24.252814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c5864cea-2730-4299-b13a-4a3ce778dbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:24.399011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5864cea-2730-4299-b13a-4a3ce778dbfa None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:24.451836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:24.509353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a9bd7-0be8-43c3-8f48-579f48b06975 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:24.509924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a9bd7-0be8-43c3-8f48-579f48b06975 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:24.510194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a9bd7-0be8-43c3-8f48-579f48b06975 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:24.510459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a9bd7-0be8-43c3-8f48-579f48b06975 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:24.512329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d59a9bd7-0be8-43c3-8f48-579f48b06975 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:24.607296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a9bd7-0be8-43c3-8f48-579f48b06975 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:24.608908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a9bd7-0be8-43c3-8f48-579f48b06975 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:24.627226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 723/1437] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:25.432982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0af4ce-75d7-4ea3-87f8-6dd55c4ebf41 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:25.433551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0af4ce-75d7-4ea3-87f8-6dd55c4ebf41 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:25.433551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0af4ce-75d7-4ea3-87f8-6dd55c4ebf41 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:25.434934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:14:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:25.440159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c28e5b-618c-49e4-a3ff-efbfd478f41e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:25.440636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c28e5b-618c-49e4-a3ff-efbfd478f41e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:25.440751 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c28e5b-618c-49e4-a3ff-efbfd478f41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:25.519521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f1c28e5b-618c-49e4-a3ff-efbfd478f41e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:25.520713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f1c28e5b-618c-49e4-a3ff-efbfd478f41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:25.523385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:25.617515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:14:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:25.769917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0af4ce-75d7-4ea3-87f8-6dd55c4ebf41 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:25.772107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-de0af4ce-75d7-4ea3-87f8-6dd55c4ebf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:25.918526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0af4ce-75d7-4ea3-87f8-6dd55c4ebf41 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:25.945973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:26.030919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c45c-a39b-4f6c-9ab6-cdfa5fc463de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:26.031478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c45c-a39b-4f6c-9ab6-cdfa5fc463de service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:26.031758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c45c-a39b-4f6c-9ab6-cdfa5fc463de service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:26.032031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c45c-a39b-4f6c-9ab6-cdfa5fc463de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:26.033730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9060c45c-a39b-4f6c-9ab6-cdfa5fc463de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:26.124189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060c45c-a39b-4f6c-9ab6-cdfa5fc463de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:26.125836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060c45c-a39b-4f6c-9ab6-cdfa5fc463de service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:26.144583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 716/1440] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:28.032412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c7862-cd7c-48d2-9a74-fb52a9b733c3 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:28.033006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c7862-cd7c-48d2-9a74-fb52a9b733c3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:28.033006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c7862-cd7c-48d2-9a74-fb52a9b733c3 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:28.034370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:28.040224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12137e14-4d27-4d66-87dd-e728f05d8c3e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:28.040681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12137e14-4d27-4d66-87dd-e728f05d8c3e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:28.040794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12137e14-4d27-4d66-87dd-e728f05d8c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:28.081284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:28.110186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-12137e14-4d27-4d66-87dd-e728f05d8c3e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:28.111590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-12137e14-4d27-4d66-87dd-e728f05d8c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:28.215562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:14:28.255350 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c7862-cd7c-48d2-9a74-fb52a9b733c3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:28.257416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a22c7862-cd7c-48d2-9a74-fb52a9b733c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:28.376918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c7862-cd7c-48d2-9a74-fb52a9b733c3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:28.540025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:28.582258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522ffb7-e590-4529-91e8-a25e88a2d462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:28.582876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ffb7-e590-4529-91e8-a25e88a2d462 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:28.583152 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ffb7-e590-4529-91e8-a25e88a2d462 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:28.583421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ffb7-e590-4529-91e8-a25e88a2d462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:28.586248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d522ffb7-e590-4529-91e8-a25e88a2d462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:28.698266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522ffb7-e590-4529-91e8-a25e88a2d462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:28.699601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522ffb7-e590-4529-91e8-a25e88a2d462 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:28.717642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 726/1443] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:30.959801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d7c7e-710c-4695-af03-07fd22f7fb5c None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:30.960355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d7c7e-710c-4695-af03-07fd22f7fb5c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:30.960355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d7c7e-710c-4695-af03-07fd22f7fb5c None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:30.961759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:30.967120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6f99-01ee-4528-b10a-d104a096c310 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:30.967524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6f99-01ee-4528-b10a-d104a096c310 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:30.967661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6f99-01ee-4528-b10a-d104a096c310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:31.032528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-cacf6f99-01ee-4528-b10a-d104a096c310 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:31.033753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cacf6f99-01ee-4528-b10a-d104a096c310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:31.046330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:31.131915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:31.170188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d7c7e-710c-4695-af03-07fd22f7fb5c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:31.172073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2c9d7c7e-710c-4695-af03-07fd22f7fb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:31.276678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d7c7e-710c-4695-af03-07fd22f7fb5c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:31.289237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:31.351207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15e570-6339-4754-999d-7b1d7c587d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:31.351827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15e570-6339-4754-999d-7b1d7c587d80 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:31.352110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15e570-6339-4754-999d-7b1d7c587d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:31.352384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15e570-6339-4754-999d-7b1d7c587d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:31.354056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8b15e570-6339-4754-999d-7b1d7c587d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:31.471322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15e570-6339-4754-999d-7b1d7c587d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:31.472662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15e570-6339-4754-999d-7b1d7c587d80 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:31.491893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 719/1446] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:33.913950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77367844-93cf-45df-8414-70bdab82fcd8 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:33.914535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77367844-93cf-45df-8414-70bdab82fcd8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:33.914535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77367844-93cf-45df-8414-70bdab82fcd8 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:33.915886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:33.921010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bba74-3b97-4974-bf1e-8f873f613d40 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:33.921440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bba74-3b97-4974-bf1e-8f873f613d40 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:33.921595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bba74-3b97-4974-bf1e-8f873f613d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:33.969282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:33.986994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f61bba74-3b97-4974-bf1e-8f873f613d40 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:33.988207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f61bba74-3b97-4974-bf1e-8f873f613d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:34.082847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:34.122958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77367844-93cf-45df-8414-70bdab82fcd8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:34.124531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-77367844-93cf-45df-8414-70bdab82fcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:34.226538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77367844-93cf-45df-8414-70bdab82fcd8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:34.267438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:34.302858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7df388-a2ac-445b-9ded-45f9c58a9d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:34.303465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7df388-a2ac-445b-9ded-45f9c58a9d74 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:34.303741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7df388-a2ac-445b-9ded-45f9c58a9d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:34.304013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7df388-a2ac-445b-9ded-45f9c58a9d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:34.305642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ea7df388-a2ac-445b-9ded-45f9c58a9d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:34.411698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7df388-a2ac-445b-9ded-45f9c58a9d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:34.413118 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7df388-a2ac-445b-9ded-45f9c58a9d74 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:34.431630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 729/1449] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:36.541499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec084bf-1f37-4245-b72a-638ae0b7bfb1 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:36.542028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec084bf-1f37-4245-b72a-638ae0b7bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:36.542028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec084bf-1f37-4245-b72a-638ae0b7bfb1 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:36.543330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:36.548538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708cfb9-fb27-46ba-be7f-8f42b7ade82f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:36.548972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708cfb9-fb27-46ba-be7f-8f42b7ade82f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:36.549083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708cfb9-fb27-46ba-be7f-8f42b7ade82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:36.616080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e708cfb9-fb27-46ba-be7f-8f42b7ade82f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:36.617280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e708cfb9-fb27-46ba-be7f-8f42b7ade82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:36.638672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:36.715158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:36.754355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec084bf-1f37-4245-b72a-638ae0b7bfb1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:36.756083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1ec084bf-1f37-4245-b72a-638ae0b7bfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:36.858729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec084bf-1f37-4245-b72a-638ae0b7bfb1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:36.878866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:36.932918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511709ef-fc92-4afe-b16c-f6b882afa43f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:36.933569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511709ef-fc92-4afe-b16c-f6b882afa43f service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:36.933854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511709ef-fc92-4afe-b16c-f6b882afa43f service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:36.934127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511709ef-fc92-4afe-b16c-f6b882afa43f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:36.935757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-511709ef-fc92-4afe-b16c-f6b882afa43f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:37.168821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511709ef-fc92-4afe-b16c-f6b882afa43f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:37.170289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511709ef-fc92-4afe-b16c-f6b882afa43f service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:37.188297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 722/1452] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:39.305121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29286889-b137-404b-afb4-ef2bffbf8982 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:39.305769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29286889-b137-404b-afb4-ef2bffbf8982 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:39.305769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29286889-b137-404b-afb4-ef2bffbf8982 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:39.307060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:14:39.312421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25348a6-9110-4889-a7ff-2e4e8908fdad None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:39.312866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25348a6-9110-4889-a7ff-2e4e8908fdad None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:39.312975 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25348a6-9110-4889-a7ff-2e4e8908fdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:39.376229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:39.379504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e25348a6-9110-4889-a7ff-2e4e8908fdad None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:39.381193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e25348a6-9110-4889-a7ff-2e4e8908fdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:39.480440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:39.518187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29286889-b137-404b-afb4-ef2bffbf8982 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:39.519814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-29286889-b137-404b-afb4-ef2bffbf8982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:39.624996 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29286889-b137-404b-afb4-ef2bffbf8982 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:39.674738 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:39.700076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6449bd4-93f7-4dc4-b5ad-ce9ccdcf9be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:39.700686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449bd4-93f7-4dc4-b5ad-ce9ccdcf9be3 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:39.700963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449bd4-93f7-4dc4-b5ad-ce9ccdcf9be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:39.701232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449bd4-93f7-4dc4-b5ad-ce9ccdcf9be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:39.702878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c6449bd4-93f7-4dc4-b5ad-ce9ccdcf9be3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:39.814694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6449bd4-93f7-4dc4-b5ad-ce9ccdcf9be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:39.816155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6449bd4-93f7-4dc4-b5ad-ce9ccdcf9be3 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:39.835197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 732/1455] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:41.972173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541972-a7a6-4a17-892b-8db073e153e6 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:41.972722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541972-a7a6-4a17-892b-8db073e153e6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:41.972722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541972-a7a6-4a17-892b-8db073e153e6 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:41.974110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:41.980308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fac5f-8531-42c8-8415-b52b518e144f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:41.980727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fac5f-8531-42c8-8415-b52b518e144f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:41.980860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fac5f-8531-42c8-8415-b52b518e144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:42.046599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-791fac5f-8531-42c8-8415-b52b518e144f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:42.047797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-791fac5f-8531-42c8-8415-b52b518e144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:42.077233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:42.147796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:14:42.187274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62541972-a7a6-4a17-892b-8db073e153e6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:42.188845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-62541972-a7a6-4a17-892b-8db073e153e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:42.291227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62541972-a7a6-4a17-892b-8db073e153e6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:42.319064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:42.365474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0d988-2a4a-4e72-8f8c-2869cb7b3e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:42.366095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0d988-2a4a-4e72-8f8c-2869cb7b3e87 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:42.366371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0d988-2a4a-4e72-8f8c-2869cb7b3e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:42.366642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0d988-2a4a-4e72-8f8c-2869cb7b3e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:42.368246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5bf0d988-2a4a-4e72-8f8c-2869cb7b3e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:42.476151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0d988-2a4a-4e72-8f8c-2869cb7b3e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:42.477626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0d988-2a4a-4e72-8f8c-2869cb7b3e87 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:42.495537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 725/1458] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:44.650201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f96eb-6224-4d71-b0fd-cf319b3574d0 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:44.650742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f96eb-6224-4d71-b0fd-cf319b3574d0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:44.650855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f96eb-6224-4d71-b0fd-cf319b3574d0 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:44.652142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:44.657706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80eb95-5440-4d90-8332-9eab7be6d13d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:44.658155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80eb95-5440-4d90-8332-9eab7be6d13d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:44.658260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80eb95-5440-4d90-8332-9eab7be6d13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:44.722600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-4f80eb95-5440-4d90-8332-9eab7be6d13d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:44.723805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4f80eb95-5440-4d90-8332-9eab7be6d13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:44.732601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:44.824828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:44.863505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f96eb-6224-4d71-b0fd-cf319b3574d0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:44.865108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0e1f96eb-6224-4d71-b0fd-cf319b3574d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:44.968956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f96eb-6224-4d71-b0fd-cf319b3574d0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:45.023881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:45.042067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9d131-7ebc-4848-8bf1-0756faf80dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:45.042686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9d131-7ebc-4848-8bf1-0756faf80dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:45.042968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9d131-7ebc-4848-8bf1-0756faf80dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:45.043243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9d131-7ebc-4848-8bf1-0756faf80dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:45.044897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fed9d131-7ebc-4848-8bf1-0756faf80dc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:45.153550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9d131-7ebc-4848-8bf1-0756faf80dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:45.154991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9d131-7ebc-4848-8bf1-0756faf80dc1 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:45.172799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 735/1461] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:47.215413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee579a-9e62-46dc-b5cc-c062a5fd0406 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:47.215972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee579a-9e62-46dc-b5cc-c062a5fd0406 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:47.215972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee579a-9e62-46dc-b5cc-c062a5fd0406 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:47.217300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:47.222688 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7baf1-4190-424a-8129-20b33bcaff88 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:47.223105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7baf1-4190-424a-8129-20b33bcaff88 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:47.223237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7baf1-4190-424a-8129-20b33bcaff88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:47.290939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-bbd7baf1-4190-424a-8129-20b33bcaff88 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:47.292124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bbd7baf1-4190-424a-8129-20b33bcaff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:47.329540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:47.388821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:47.426540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee579a-9e62-46dc-b5cc-c062a5fd0406 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:47.428786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f0ee579a-9e62-46dc-b5cc-c062a5fd0406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:47.530492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee579a-9e62-46dc-b5cc-c062a5fd0406 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:47.569994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:47.604905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93b523-beac-471e-b0cd-2708b5942d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:47.605535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93b523-beac-471e-b0cd-2708b5942d4c service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:47.605812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93b523-beac-471e-b0cd-2708b5942d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:47.606082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93b523-beac-471e-b0cd-2708b5942d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:47.607681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0c93b523-beac-471e-b0cd-2708b5942d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:47.714595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93b523-beac-471e-b0cd-2708b5942d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:47.715978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93b523-beac-471e-b0cd-2708b5942d4c service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:47.735135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 728/1464] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:49.763596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c853c-1752-473b-a7c2-8b4cf830a19c None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:49.764479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c853c-1752-473b-a7c2-8b4cf830a19c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:49.764665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c853c-1752-473b-a7c2-8b4cf830a19c None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:49.766104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:49.771319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95c8e-53b8-4c72-9860-e9d24a9f2510 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:49.771927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95c8e-53b8-4c72-9860-e9d24a9f2510 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:49.771927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95c8e-53b8-4c72-9860-e9d24a9f2510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:49.837116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a1e95c8e-53b8-4c72-9860-e9d24a9f2510 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:49.838378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a1e95c8e-53b8-4c72-9860-e9d24a9f2510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:49.856558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:49.933899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:49.971421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c853c-1752-473b-a7c2-8b4cf830a19c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:49.972951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a93c853c-1752-473b-a7c2-8b4cf830a19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:50.076354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c853c-1752-473b-a7c2-8b4cf830a19c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:50.143876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:50.151759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831e2cb-686d-426d-88c3-8e1c35a1ce0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:50.152387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e2cb-686d-426d-88c3-8e1c35a1ce0d service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:50.152666 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e2cb-686d-426d-88c3-8e1c35a1ce0d service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:50.152939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e2cb-686d-426d-88c3-8e1c35a1ce0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:50.154601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-1831e2cb-686d-426d-88c3-8e1c35a1ce0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:50.260986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831e2cb-686d-426d-88c3-8e1c35a1ce0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:50.262685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831e2cb-686d-426d-88c3-8e1c35a1ce0d service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:50.280742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 738/1467] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:14:52.336459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fcaa9-8eda-4c87-aa02-754f02b019c6 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:52.336993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fcaa9-8eda-4c87-aa02-754f02b019c6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:52.336993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fcaa9-8eda-4c87-aa02-754f02b019c6 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:52.338417 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:14:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:14:52.343838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb9562-181b-4dd3-b44e-32dbf32bc0dd None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:52.344267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb9562-181b-4dd3-b44e-32dbf32bc0dd None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:52.344374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb9562-181b-4dd3-b44e-32dbf32bc0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:52.414043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-fecb9562-181b-4dd3-b44e-32dbf32bc0dd None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:14:52.415191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fecb9562-181b-4dd3-b44e-32dbf32bc0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:52.465355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:52.516530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:14:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:14:52.554340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fcaa9-8eda-4c87-aa02-754f02b019c6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:14:52.555973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3a2fcaa9-8eda-4c87-aa02-754f02b019c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:52.657370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fcaa9-8eda-4c87-aa02-754f02b019c6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:14:52.703892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:14:52.730072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b4560-5ef7-4930-9aca-34a97804d876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:14:52.730693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b4560-5ef7-4930-9aca-34a97804d876 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:14:52.730973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b4560-5ef7-4930-9aca-34a97804d876 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:14:52.731247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b4560-5ef7-4930-9aca-34a97804d876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:14:52.732867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-904b4560-5ef7-4930-9aca-34a97804d876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:14:52.850671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b4560-5ef7-4930-9aca-34a97804d876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:14:52.852112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b4560-5ef7-4930-9aca-34a97804d876 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:14:52.870310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 731/1470] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:14:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:15:02.514778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=22 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:02.867820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=66 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:12.478766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03812480-49ca-4272-ae36-524d377a1c2c None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:12.479277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03812480-49ca-4272-ae36-524d377a1c2c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:12.479277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03812480-49ca-4272-ae36-524d377a1c2c None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:12.480580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 740/1471] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:15:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:15:12.485812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c664a2-b8f3-4968-a28e-2ed84ff03017 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:12.486215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c664a2-b8f3-4968-a28e-2ed84ff03017 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:12.486369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c664a2-b8f3-4968-a28e-2ed84ff03017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:12.552430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-92c664a2-b8f3-4968-a28e-2ed84ff03017 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:15:12.553669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-92c664a2-b8f3-4968-a28e-2ed84ff03017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:12.648010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 732/1472] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:15:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:15:12.732936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ce572-1612-440b-bcba-54678b31f695 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:12.733162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ce572-1612-440b-bcba-54678b31f695 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:12.733456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ce572-1612-440b-bcba-54678b31f695 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:12.734789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 741/1473] 10.222.0.109 () {54 vars in 971 bytes} [Thu Aug 4 22:15:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:15:12.739670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5fd6-6c1e-44be-83fc-a27ae3ff4713 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:12.739959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5fd6-6c1e-44be-83fc-a27ae3ff4713 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:12.740263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5fd6-6c1e-44be-83fc-a27ae3ff4713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:12.816210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e31d5fd6-6c1e-44be-83fc-a27ae3ff4713 None None] MFA Rules not processed for user `33183e28de8f487f82f92f0151e7c73c`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:15:12.817967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e31d5fd6-6c1e-44be-83fc-a27ae3ff4713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:12.838790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:12.894407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 733/1474] 10.222.0.109 () {58 vars in 1068 bytes} [Thu Aug 4 22:15:12 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:15:12.902679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ce572-1612-440b-bcba-54678b31f695 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:15:12.904216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-792ce572-1612-440b-bcba-54678b31f695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:13.006247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ce572-1612-440b-bcba-54678b31f695 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:15:13.077671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7a934-62f8-43bc-b26a-434126ddb0ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33183e28de8f487f82f92f0151e7c73c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:15:13.078281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a934-62f8-43bc-b26a-434126ddb0ac service glance] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:13.078554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a934-62f8-43bc-b26a-434126ddb0ac service glance] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:13.078816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a934-62f8-43bc-b26a-434126ddb0ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:13.080402 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e6f7a934-62f8-43bc-b26a-434126ddb0ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:13.137937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:13.193934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7a934-62f8-43bc-b26a-434126ddb0ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:15:13.195425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7a934-62f8-43bc-b26a-434126ddb0ac service glance] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:15:13.214456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 742/1475] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:15:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:15:15.306902 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7ef9-0127-4685-848d-759e3833622a None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:15.307439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7ef9-0127-4685-848d-759e3833622a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:15.307439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7ef9-0127-4685-848d-759e3833622a None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:15.308773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 734/1476] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:15:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:15:15.314158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f0ce3-759f-464d-808f-376ec5969ae8 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:15.314588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f0ce3-759f-464d-808f-376ec5969ae8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:15.314711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f0ce3-759f-464d-808f-376ec5969ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:15.379713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-391f0ce3-759f-464d-808f-376ec5969ae8 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:15:15.380919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-391f0ce3-759f-464d-808f-376ec5969ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:15.478912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 743/1477] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:15:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:15:15.501858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b7ef9-0127-4685-848d-759e3833622a None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:15:15.503399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fc8b7ef9-0127-4685-848d-759e3833622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:15.632116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b7ef9-0127-4685-848d-759e3833622a None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:15:15.681875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:15.726604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4338f82-dcf4-42da-ab57-17823b78e053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:15:15.727235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338f82-dcf4-42da-ab57-17823b78e053 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:15.727493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338f82-dcf4-42da-ab57-17823b78e053 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:15.727754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338f82-dcf4-42da-ab57-17823b78e053 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:15.728437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4338f82-dcf4-42da-ab57-17823b78e053 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:15:15.730195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING py.warnings [None req-c4338f82-dcf4-42da-ab57-17823b78e053 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:15:15.730195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: warnings.warn(msg) Aug 04 22:15:15.730195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: Aug 04 22:15:15.731320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4338f82-dcf4-42da-ab57-17823b78e053 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:15:15.750943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 735/1478] 10.222.0.109 () {56 vars in 1113 bytes} [Thu Aug 4 22:15:15 2022] GET /identity/v3/projects => generated 2562 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:15:17.807561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb78e-2941-4cfe-b5e2-b699148acd98 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:17.808178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb78e-2941-4cfe-b5e2-b699148acd98 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:17.808178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb78e-2941-4cfe-b5e2-b699148acd98 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:17.809491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 744/1479] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:15:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:15:17.814907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66421688-cd00-41d1-a9c2-0a173f7759d6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:17.815316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66421688-cd00-41d1-a9c2-0a173f7759d6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:17.815460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66421688-cd00-41d1-a9c2-0a173f7759d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:17.886412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-66421688-cd00-41d1-a9c2-0a173f7759d6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:15:17.887661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-66421688-cd00-41d1-a9c2-0a173f7759d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:17.983158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 736/1480] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:15:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:15:18.027062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eb78e-2941-4cfe-b5e2-b699148acd98 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:15:18.028745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-422eb78e-2941-4cfe-b5e2-b699148acd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:18.038816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:18.177383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eb78e-2941-4cfe-b5e2-b699148acd98 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:15:18.257052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:18.290177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c83ec-1cc8-4461-a32a-3691d682a94f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:15:18.290794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c83ec-1cc8-4461-a32a-3691d682a94f service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:18.291071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c83ec-1cc8-4461-a32a-3691d682a94f service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:18.291342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c83ec-1cc8-4461-a32a-3691d682a94f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:18.292962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8c3c83ec-1cc8-4461-a32a-3691d682a94f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:18.382336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c83ec-1cc8-4461-a32a-3691d682a94f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:15:18.383848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c83ec-1cc8-4461-a32a-3691d682a94f service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:15:18.401926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 745/1481] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:15:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:15:20.592972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bf1c-4042-4278-9bb3-812e5d91b085 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:20.593548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bf1c-4042-4278-9bb3-812e5d91b085 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:20.593548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bf1c-4042-4278-9bb3-812e5d91b085 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:20.594899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 737/1482] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:15:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:15:20.600176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323488b-a5de-4617-9e16-a5ebb15f5cd0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:20.600634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323488b-a5de-4617-9e16-a5ebb15f5cd0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:20.600750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323488b-a5de-4617-9e16-a5ebb15f5cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:20.665765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4323488b-a5de-4617-9e16-a5ebb15f5cd0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:15:20.666973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4323488b-a5de-4617-9e16-a5ebb15f5cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:20.712950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:20.763754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 746/1483] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:15:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:15:20.799722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16bf1c-4042-4278-9bb3-812e5d91b085 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:15:20.801836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ce16bf1c-4042-4278-9bb3-812e5d91b085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:20.824795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:20.903955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16bf1c-4042-4278-9bb3-812e5d91b085 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:15:20.977391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab944d27-9a8d-41a9-a5aa-91945d779941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:15:20.978039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944d27-9a8d-41a9-a5aa-91945d779941 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:20.978319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944d27-9a8d-41a9-a5aa-91945d779941 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:20.978589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944d27-9a8d-41a9-a5aa-91945d779941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:20.980222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ab944d27-9a8d-41a9-a5aa-91945d779941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:21.054607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:21.088900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab944d27-9a8d-41a9-a5aa-91945d779941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:15:21.090403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab944d27-9a8d-41a9-a5aa-91945d779941 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:15:21.110282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 738/1484] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:15:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:15:23.193665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9aca3-6358-4970-bdab-593b294eeb1a None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:23.194271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9aca3-6358-4970-bdab-593b294eeb1a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:23.194271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9aca3-6358-4970-bdab-593b294eeb1a None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:23.195465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 747/1485] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:15:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:15:23.200876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc607e6-4c8f-471e-8f87-acb150f6efe4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:23.201320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc607e6-4c8f-471e-8f87-acb150f6efe4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:23.201508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc607e6-4c8f-471e-8f87-acb150f6efe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:23.271619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ccc607e6-4c8f-471e-8f87-acb150f6efe4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:15:23.272910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ccc607e6-4c8f-471e-8f87-acb150f6efe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:23.385662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 739/1486] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:15:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:15:23.426074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9aca3-6358-4970-bdab-593b294eeb1a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:15:23.427681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-20d9aca3-6358-4970-bdab-593b294eeb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:23.530668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9aca3-6358-4970-bdab-593b294eeb1a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:15:23.603192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:23.605056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a0c7f-0a6d-44f0-a413-56b19ff9eeb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:15:23.605819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a0c7f-0a6d-44f0-a413-56b19ff9eeb3 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:23.606321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a0c7f-0a6d-44f0-a413-56b19ff9eeb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:23.606759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a0c7f-0a6d-44f0-a413-56b19ff9eeb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:23.608574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-277a0c7f-0a6d-44f0-a413-56b19ff9eeb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:23.717791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a0c7f-0a6d-44f0-a413-56b19ff9eeb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:15:23.719214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a0c7f-0a6d-44f0-a413-56b19ff9eeb3 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:15:23.736882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 748/1487] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:15:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:15:25.828484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b96f6e-380e-45b6-9d0d-6586665a8c20 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:25.829019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b96f6e-380e-45b6-9d0d-6586665a8c20 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:25.829140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b96f6e-380e-45b6-9d0d-6586665a8c20 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:25.830444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 740/1488] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:15:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:15:25.837385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60125e28-1f2b-47d0-9253-a27389c32663 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:25.837822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60125e28-1f2b-47d0-9253-a27389c32663 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:25.837946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60125e28-1f2b-47d0-9253-a27389c32663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:25.903938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-60125e28-1f2b-47d0-9253-a27389c32663 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:15:25.905174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-60125e28-1f2b-47d0-9253-a27389c32663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:25.958115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:26.003123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 749/1489] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:15:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:15:26.044840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b96f6e-380e-45b6-9d0d-6586665a8c20 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:15:26.046439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-72b96f6e-380e-45b6-9d0d-6586665a8c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:26.082030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:26.150644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b96f6e-380e-45b6-9d0d-6586665a8c20 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:15:26.222656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08155282-c0ab-418a-bb5e-648ebf531e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:15:26.223411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155282-c0ab-418a-bb5e-648ebf531e07 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:26.223684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155282-c0ab-418a-bb5e-648ebf531e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:26.223949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155282-c0ab-418a-bb5e-648ebf531e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:26.225588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-08155282-c0ab-418a-bb5e-648ebf531e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:26.306972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:26.340241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08155282-c0ab-418a-bb5e-648ebf531e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:15:26.341774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08155282-c0ab-418a-bb5e-648ebf531e07 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:15:26.360707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 741/1490] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:15:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:15:28.629370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed9d6e-a523-46d5-878b-edb66a2d0ab9 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:28.629927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed9d6e-a523-46d5-878b-edb66a2d0ab9 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:28.629927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed9d6e-a523-46d5-878b-edb66a2d0ab9 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:28.631302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 750/1491] 10.222.0.109 () {54 vars in 932 bytes} [Thu Aug 4 22:15:28 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:15:28.636781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de88a13-8c9d-4bcb-a61e-b3f240785850 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:28.637188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de88a13-8c9d-4bcb-a61e-b3f240785850 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:28.637333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de88a13-8c9d-4bcb-a61e-b3f240785850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:28.705274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0de88a13-8c9d-4bcb-a61e-b3f240785850 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:15:28.707291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0de88a13-8c9d-4bcb-a61e-b3f240785850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:28.803152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 742/1492] 10.222.0.109 () {58 vars in 1029 bytes} [Thu Aug 4 22:15:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:15:28.843863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed9d6e-a523-46d5-878b-edb66a2d0ab9 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:15:28.845459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e6ed9d6e-a523-46d5-878b-edb66a2d0ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:28.955361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed9d6e-a523-46d5-878b-edb66a2d0ab9 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:15:29.034248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195b6e7-c542-4ab3-a2d8-0724f39a7923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:15:29.034980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195b6e7-c542-4ab3-a2d8-0724f39a7923 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:15:29.035257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195b6e7-c542-4ab3-a2d8-0724f39a7923 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:15:29.035521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195b6e7-c542-4ab3-a2d8-0724f39a7923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:15:29.037133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a195b6e7-c542-4ab3-a2d8-0724f39a7923 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:15:29.046801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:29.149621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195b6e7-c542-4ab3-a2d8-0724f39a7923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:15:29.150975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195b6e7-c542-4ab3-a2d8-0724f39a7923 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:15:29.168672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 751/1493] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:15:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:15:38.801279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=82 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:15:39.166405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=50 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:41.832959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4081d-737c-4d85-8604-6363e823da83 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:41.833571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4081d-737c-4d85-8604-6363e823da83 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:41.833571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4081d-737c-4d85-8604-6363e823da83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:41.912937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-02c4081d-737c-4d85-8604-6363e823da83 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:16:41.914174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-02c4081d-737c-4d85-8604-6363e823da83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:42.009690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 743/1494] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:16:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:16:42.018560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511001a-bd4a-470c-92ee-4403bcbe5d9e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:42.019087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511001a-bd4a-470c-92ee-4403bcbe5d9e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:42.019087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511001a-bd4a-470c-92ee-4403bcbe5d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:42.094576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6511001a-bd4a-470c-92ee-4403bcbe5d9e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:16:42.095683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6511001a-bd4a-470c-92ee-4403bcbe5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:42.192193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 752/1495] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:16:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:16:42.196955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4081d-737c-4d85-8604-6363e823da83 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:42.198231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-02c4081d-737c-4d85-8604-6363e823da83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:42.210463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4081d-737c-4d85-8604-6363e823da83 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:42.272248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:42.307073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:42.307693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:42.307972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:42.308242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:42.308930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:16:42.309231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:42.310287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING py.warnings [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:16:42.310287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: warnings.warn(msg) Aug 04 22:16:42.310287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: Aug 04 22:16:42.311208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:42.322260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 744/1496] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:16:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:16:42.331057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511001a-bd4a-470c-92ee-4403bcbe5d9e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:42.339973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511001a-bd4a-470c-92ee-4403bcbe5d9e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:42.400159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:42.433839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:42.434451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:42.434722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:42.434986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:42.435641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:42.437379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WARNING py.warnings [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:16:42.437379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: warnings.warn(msg) Aug 04 22:16:42.437379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: Aug 04 22:16:42.438575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:42.497748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 753/1497] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:16:42 2022] POST /identity/v3/projects => generated 380 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:16:42.502330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:42.503889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:42.538325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a6e2-f926-45f5-83fa-a1c681a4fcfc admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:42.631580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:42.632227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:42.632505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:42.632777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:42.633470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:42.634757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING py.warnings [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:16:42.634757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: warnings.warn(msg) Aug 04 22:16:42.634757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: Aug 04 22:16:42.635975 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:42.679528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 745/1498] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:16:42 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:16:42.683547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:42.685043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:42.754382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf88188-0144-49d9-bb26-eafcb2e742e5 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:42.838236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:42.850125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:42.850742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:42.851016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:42.851282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:42.851963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:42.853137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WARNING py.warnings [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:16:42.853137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: warnings.warn(msg) Aug 04 22:16:42.853137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: Aug 04 22:16:42.854244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:42.866648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 754/1499] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:16:42 2022] GET /identity/v3/roles => generated 1427 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:16:42.870710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:42.872219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:42.884432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46562bce-42c1-4fe1-b41a-060dab8e8fcd admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:42.909791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:42.979799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63874495-25c2-4e66-a158-812ae71a4447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:42.980347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874495-25c2-4e66-a158-812ae71a4447 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:42.980629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874495-25c2-4e66-a158-812ae71a4447 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:42.980898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874495-25c2-4e66-a158-812ae71a4447 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:42.981621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63874495-25c2-4e66-a158-812ae71a4447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:42.982647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING py.warnings [None req-63874495-25c2-4e66-a158-812ae71a4447 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:16:42.982647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: warnings.warn(msg) Aug 04 22:16:42.982647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: Aug 04 22:16:42.983736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63874495-25c2-4e66-a158-812ae71a4447 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:42.995784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 746/1500] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:16:42 2022] GET /identity/v3/roles => generated 1427 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:16:43.000808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:43.002369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:43.036029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051fa76-6ccd-4a0c-814b-5a4eea8c3f52 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:43.130308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4d45b-0f95-4d8f-9ab5-1d31727dc0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:43.130977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d45b-0f95-4d8f-9ab5-1d31727dc0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:43.131248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d45b-0f95-4d8f-9ab5-1d31727dc0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:43.131516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d45b-0f95-4d8f-9ab5-1d31727dc0d8 admin admin] PATH_INFO: `/v3/projects/c322c26d1cac41c1bc9f3f6129d4257f/users/7ce0beacbcfe41d2acaf317da7c6da3e/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:43.153344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4d45b-0f95-4d8f-9ab5-1d31727dc0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c322c26d1cac41c1bc9f3f6129d4257f, user_id=7ce0beacbcfe41d2acaf317da7c6da3e, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:43.154404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WARNING py.warnings [None req-c3c4d45b-0f95-4d8f-9ab5-1d31727dc0d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:16:43.154404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: warnings.warn(msg) Aug 04 22:16:43.154404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: Aug 04 22:16:43.156566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4d45b-0f95-4d8f-9ab5-1d31727dc0d8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:43.180635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 755/1501] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:16:42 2022] PUT /identity/v3/projects/c322c26d1cac41c1bc9f3f6129d4257f/users/7ce0beacbcfe41d2acaf317da7c6da3e/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:16:43.193626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf9b09-cf3b-4516-b0c6-fec6f6e77978 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:43.194021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf9b09-cf3b-4516-b0c6-fec6f6e77978 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:43.194365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf9b09-cf3b-4516-b0c6-fec6f6e77978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:43.252146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8baf9b09-cf3b-4516-b0c6-fec6f6e77978 None None] MFA Rules not processed for user `7ce0beacbcfe41d2acaf317da7c6da3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:16:43.253296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8baf9b09-cf3b-4516-b0c6-fec6f6e77978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:43.333493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 747/1502] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:16:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:16:43.347287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635f803-fc86-400d-9131-bed8754b0c5b None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:43.347677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635f803-fc86-400d-9131-bed8754b0c5b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:43.348037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635f803-fc86-400d-9131-bed8754b0c5b None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:43.349610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 756/1503] 10.222.0.109 () {54 vars in 969 bytes} [Thu Aug 4 22:16:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:16:43.354267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90affdcf-5737-4f3c-b6d5-a7592a84cf4c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:43.354559 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90affdcf-5737-4f3c-b6d5-a7592a84cf4c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:43.354826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90affdcf-5737-4f3c-b6d5-a7592a84cf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:43.385347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:43.429719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-90affdcf-5737-4f3c-b6d5-a7592a84cf4c None None] MFA Rules not processed for user `1d0422cd82114045a9fff7df29bc3906`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:16:43.430938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-90affdcf-5737-4f3c-b6d5-a7592a84cf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:43.505893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 748/1504] 10.222.0.109 () {58 vars in 1066 bytes} [Thu Aug 4 22:16:43 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:16:43.513903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635f803-fc86-400d-9131-bed8754b0c5b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:43.516616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0635f803-fc86-400d-9131-bed8754b0c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:43.568896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:43.624528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635f803-fc86-400d-9131-bed8754b0c5b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:43.696340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:43.696963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:43.697246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:43.697580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:43.699245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:43.798335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:43.799689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:43.803231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:43.819030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 757/1505] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:16:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:16:43.864716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa846db3-1b5f-425c-89cd-e23fb8ce2dce None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:43.865005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa846db3-1b5f-425c-89cd-e23fb8ce2dce None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:43.865287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa846db3-1b5f-425c-89cd-e23fb8ce2dce None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:43.866633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 749/1506] 10.222.0.109 () {52 vars in 835 bytes} [Thu Aug 4 22:16:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:16:43.870560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:43.871795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:43.970856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967dbe7-bb60-4ec7-8064-98bb57dc94ae service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:44.064231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:44.064853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:44.065053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:44.065322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] PATH_INFO: `/v3/users/7ce0beacbcfe41d2acaf317da7c6da3e` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:44.077539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce0beacbcfe41d2acaf317da7c6da3e)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:44.078568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WARNING py.warnings [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:16:44.078568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: warnings.warn(msg) Aug 04 22:16:44.078568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: Aug 04 22:16:44.079777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:44.089161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:44.147311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce0beacbcfe41d2acaf317da7c6da3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:44.161049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce0beacbcfe41d2acaf317da7c6da3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:44.169211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce0beacbcfe41d2acaf317da7c6da3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:44.169828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce0beacbcfe41d2acaf317da7c6da3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:44.192645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce0beacbcfe41d2acaf317da7c6da3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:44.198248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 758/1507] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:16:43 2022] DELETE /identity/v3/users/7ce0beacbcfe41d2acaf317da7c6da3e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:16:44.202804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e7fa-e98f-456d-a84f-94dc7f94c12e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:44.203307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e7fa-e98f-456d-a84f-94dc7f94c12e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:44.203416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36e7fa-e98f-456d-a84f-94dc7f94c12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:44.260704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2b36e7fa-e98f-456d-a84f-94dc7f94c12e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:16:44.262020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2b36e7fa-e98f-456d-a84f-94dc7f94c12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:44.335639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:44.348506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 750/1508] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:16:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:16:44.358359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:44.360112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:44.461521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf28675-167d-48dc-ac0c-a1d4597c6ee9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:44.514007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:44.535315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb7d12-72e6-4362-bf3f-328b21dd5ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:44.535931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7d12-72e6-4362-bf3f-328b21dd5ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:44.536206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7d12-72e6-4362-bf3f-328b21dd5ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:44.536474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7d12-72e6-4362-bf3f-328b21dd5ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:44.538111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-dceb7d12-72e6-4362-bf3f-328b21dd5ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:44.643814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb7d12-72e6-4362-bf3f-328b21dd5ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:44.645201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb7d12-72e6-4362-bf3f-328b21dd5ba2 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:44.663363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 759/1509] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:16:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:16:45.115894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36e7fa-e98f-456d-a84f-94dc7f94c12e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:16:45.117183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2b36e7fa-e98f-456d-a84f-94dc7f94c12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:16:45.129844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36e7fa-e98f-456d-a84f-94dc7f94c12e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:16:45.232163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:16:45.232827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:16:45.233077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:16:45.233933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] PATH_INFO: `/v3/projects/c322c26d1cac41c1bc9f3f6129d4257f` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:16:45.243546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c322c26d1cac41c1bc9f3f6129d4257f)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:16:45.244586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING py.warnings [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:16:45.244586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: warnings.warn(msg) Aug 04 22:16:45.244586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: Aug 04 22:16:45.245821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:16:45.302555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c322c26d1cac41c1bc9f3f6129d4257f', 'request_id': 'req-63969484-5cc3-4b83-9dc0-36c9466ddbfc'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:45.311796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c322c26d1cac41c1bc9f3f6129d4257f', 'request_id': 'req-63969484-5cc3-4b83-9dc0-36c9466ddbfc'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:45.318204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c322c26d1cac41c1bc9f3f6129d4257f', 'request_id': 'req-63969484-5cc3-4b83-9dc0-36c9466ddbfc'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:45.349734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] The token cache is being invalidate because project c322c26d1cac41c1bc9f3f6129d4257f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:16:45.350022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:45.350552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-63969484-5cc3-4b83-9dc0-36c9466ddbfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:16:45.351901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 751/1510] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:16:45 2022] DELETE /identity/v3/projects/c322c26d1cac41c1bc9f3f6129d4257f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:16:54.661147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=62 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:16:55.347707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:18:15.103558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afbef4-3c46-4ce8-87d4-9d4b83beb86a None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:15.104133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afbef4-3c46-4ce8-87d4-9d4b83beb86a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:15.104133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afbef4-3c46-4ce8-87d4-9d4b83beb86a None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:15.105524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:18:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:15.110476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd90e82-72e0-42e5-9512-d06608d57259 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:15.110937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd90e82-72e0-42e5-9512-d06608d57259 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:15.111501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd90e82-72e0-42e5-9512-d06608d57259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:15.193654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-dfd90e82-72e0-42e5-9512-d06608d57259 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:15.194869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-dfd90e82-72e0-42e5-9512-d06608d57259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:15.295705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:18:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:18:15.454017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afbef4-3c46-4ce8-87d4-9d4b83beb86a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:15.456259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-14afbef4-3c46-4ce8-87d4-9d4b83beb86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:15.576324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afbef4-3c46-4ce8-87d4-9d4b83beb86a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:15.582855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9828dff-4cd3-4995-a16c-6db4381a9e55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:15.583469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9828dff-4cd3-4995-a16c-6db4381a9e55 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:15.583747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9828dff-4cd3-4995-a16c-6db4381a9e55 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:15.584019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9828dff-4cd3-4995-a16c-6db4381a9e55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:15.592002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9828dff-4cd3-4995-a16c-6db4381a9e55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:15.596522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9828dff-4cd3-4995-a16c-6db4381a9e55 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:15.605915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 761/1513] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:16.442745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a276cd8-5585-4d5d-9a4b-06e6e0f465b2 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:16.443292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a276cd8-5585-4d5d-9a4b-06e6e0f465b2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:16.443292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a276cd8-5585-4d5d-9a4b-06e6e0f465b2 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:16.444561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:18:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:16.449435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148592d3-9c4c-4479-8945-129ac72e3d2e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:16.449882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148592d3-9c4c-4479-8945-129ac72e3d2e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:16.449983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148592d3-9c4c-4479-8945-129ac72e3d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:16.470595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-148592d3-9c4c-4479-8945-129ac72e3d2e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:16.471787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-148592d3-9c4c-4479-8945-129ac72e3d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:16.479677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:18:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:16.631758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a276cd8-5585-4d5d-9a4b-06e6e0f465b2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:16.646486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a276cd8-5585-4d5d-9a4b-06e6e0f465b2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:16.651787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039dc72-c8af-4534-95a1-cb30cca69289 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:16.652387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039dc72-c8af-4534-95a1-cb30cca69289 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:16.652663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039dc72-c8af-4534-95a1-cb30cca69289 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:16.652933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039dc72-c8af-4534-95a1-cb30cca69289 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:16.660947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039dc72-c8af-4534-95a1-cb30cca69289 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:16.662563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039dc72-c8af-4534-95a1-cb30cca69289 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:16.672652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 754/1516] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:17.103408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042f5d0-c840-4e3b-8400-a3bc4cf787d7 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:17.103795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042f5d0-c840-4e3b-8400-a3bc4cf787d7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:17.104131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042f5d0-c840-4e3b-8400-a3bc4cf787d7 None None] PATH_INFO: `/v3` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:17.105505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 763/1517] 10.222.0.42 () {54 vars in 929 bytes} [Thu Aug 4 22:18:17 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:18:17.112481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd01b-0c19-45c7-a8ab-cbaeb5fc960b None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:17.112771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd01b-0c19-45c7-a8ab-cbaeb5fc960b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:17.113047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd01b-0c19-45c7-a8ab-cbaeb5fc960b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:17.172713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6efcd01b-0c19-45c7-a8ab-cbaeb5fc960b None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:17.173958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6efcd01b-0c19-45c7-a8ab-cbaeb5fc960b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:17.190412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:18:17.245977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 755/1518] 10.222.0.42 () {58 vars in 1019 bytes} [Thu Aug 4 22:18:17 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:26.256667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309ad37-90b2-41f1-b041-b57a76560460 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:26.257196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309ad37-90b2-41f1-b041-b57a76560460 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:26.257316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309ad37-90b2-41f1-b041-b57a76560460 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:26.258586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 764/1519] 10.222.0.109 () {54 vars in 966 bytes} [Thu Aug 4 22:18:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:26.263772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee484db-d1fa-4a18-8fa0-81c27c2219bd None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:26.264209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee484db-d1fa-4a18-8fa0-81c27c2219bd None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:26.264359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee484db-d1fa-4a18-8fa0-81c27c2219bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:26.283920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-1ee484db-d1fa-4a18-8fa0-81c27c2219bd None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:26.285698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1ee484db-d1fa-4a18-8fa0-81c27c2219bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:26.294264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 756/1520] 10.222.0.109 () {58 vars in 1063 bytes} [Thu Aug 4 22:18:26 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:26.301737 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309ad37-90b2-41f1-b041-b57a76560460 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:26.314877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309ad37-90b2-41f1-b041-b57a76560460 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:26.320476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441aaa35-f811-490d-a2b8-89267bcc5bb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:26.321087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aaa35-f811-490d-a2b8-89267bcc5bb4 service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:26.321367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aaa35-f811-490d-a2b8-89267bcc5bb4 service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:26.321668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aaa35-f811-490d-a2b8-89267bcc5bb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:26.330601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441aaa35-f811-490d-a2b8-89267bcc5bb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:26.333079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441aaa35-f811-490d-a2b8-89267bcc5bb4 service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:26.343882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 765/1521] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:26 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:28.061331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67a5af-9eee-42e4-9ed6-fa9767061922 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:28.061931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67a5af-9eee-42e4-9ed6-fa9767061922 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:28.061931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67a5af-9eee-42e4-9ed6-fa9767061922 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:28.063205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:18:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:28.068874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24508414-2f88-4360-b126-033a50a9e397 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:28.069284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24508414-2f88-4360-b126-033a50a9e397 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:28.069445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24508414-2f88-4360-b126-033a50a9e397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:28.088775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-24508414-2f88-4360-b126-033a50a9e397 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:28.089993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-24508414-2f88-4360-b126-033a50a9e397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:28.099015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:18:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:28.259299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67a5af-9eee-42e4-9ed6-fa9767061922 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:28.274515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67a5af-9eee-42e4-9ed6-fa9767061922 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:28.280005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6892f38-9987-4430-9872-cec7cf8a075a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:28.280622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6892f38-9987-4430-9872-cec7cf8a075a service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:28.280906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6892f38-9987-4430-9872-cec7cf8a075a service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:28.281186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6892f38-9987-4430-9872-cec7cf8a075a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:28.290241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6892f38-9987-4430-9872-cec7cf8a075a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:28.291839 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6892f38-9987-4430-9872-cec7cf8a075a service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:28.301204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 758/1524] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:29.115928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437d6f-d0bd-4e3a-a2ea-e3d89979847c None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:29.116419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437d6f-d0bd-4e3a-a2ea-e3d89979847c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:29.116525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437d6f-d0bd-4e3a-a2ea-e3d89979847c None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:29.117857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 22:18:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:29.122766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712a0ef-6940-4404-ae7c-4313c38bf8a7 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:29.123160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712a0ef-6940-4404-ae7c-4313c38bf8a7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:29.123317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712a0ef-6940-4404-ae7c-4313c38bf8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:29.143216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9712a0ef-6940-4404-ae7c-4313c38bf8a7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:29.145161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9712a0ef-6940-4404-ae7c-4313c38bf8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:29.153114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 22:18:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:29.305466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b437d6f-d0bd-4e3a-a2ea-e3d89979847c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:29.320394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b437d6f-d0bd-4e3a-a2ea-e3d89979847c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:29.325856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb32957-0d85-4354-8095-cc59b023bba2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:29.326462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb32957-0d85-4354-8095-cc59b023bba2 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:29.326742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb32957-0d85-4354-8095-cc59b023bba2 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:29.327013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb32957-0d85-4354-8095-cc59b023bba2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:29.335325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb32957-0d85-4354-8095-cc59b023bba2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:29.336893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb32957-0d85-4354-8095-cc59b023bba2 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:29.346305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 768/1527] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:33.074832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9487-293f-4205-aa08-89fd7c7f4e5e None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:33.075379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9487-293f-4205-aa08-89fd7c7f4e5e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:33.075379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9487-293f-4205-aa08-89fd7c7f4e5e None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:33.076754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:18:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:33.082098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953857-e4ea-4db4-9f2e-121fd59c78ae None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:33.082618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953857-e4ea-4db4-9f2e-121fd59c78ae None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:33.082900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953857-e4ea-4db4-9f2e-121fd59c78ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:33.102617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-d5953857-e4ea-4db4-9f2e-121fd59c78ae None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:33.103831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d5953857-e4ea-4db4-9f2e-121fd59c78ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:33.111816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:18:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:33.184875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f9487-293f-4205-aa08-89fd7c7f4e5e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:33.186633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f66f9487-293f-4205-aa08-89fd7c7f4e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:33.265052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f9487-293f-4205-aa08-89fd7c7f4e5e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:33.268505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd3b38-ab5d-489b-9acd-5ac9ceb18196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:33.269110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3b38-ab5d-489b-9acd-5ac9ceb18196 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:33.269387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3b38-ab5d-489b-9acd-5ac9ceb18196 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:33.269713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3b38-ab5d-489b-9acd-5ac9ceb18196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:33.279282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd3b38-ab5d-489b-9acd-5ac9ceb18196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:33.280764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd3b38-ab5d-489b-9acd-5ac9ceb18196 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:33.292230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 761/1530] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:35.278304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d83ef-b797-4263-b450-baf0b54bb628 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:35.278850 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d83ef-b797-4263-b450-baf0b54bb628 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:35.278850 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d83ef-b797-4263-b450-baf0b54bb628 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:35.280122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:18:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:35.286669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4a756-3e54-4356-9497-979e181ae032 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:35.287105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4a756-3e54-4356-9497-979e181ae032 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:35.287227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4a756-3e54-4356-9497-979e181ae032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:35.306680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-17a4a756-3e54-4356-9497-979e181ae032 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:35.307872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-17a4a756-3e54-4356-9497-979e181ae032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:35.315713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:18:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:35.353983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d83ef-b797-4263-b450-baf0b54bb628 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:35.355675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-1e3d83ef-b797-4263-b450-baf0b54bb628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:35.368577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d83ef-b797-4263-b450-baf0b54bb628 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:35.372159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa8ec6-98eb-4281-97cb-fd38530f7b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:35.372763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa8ec6-98eb-4281-97cb-fd38530f7b7f service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:35.373046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa8ec6-98eb-4281-97cb-fd38530f7b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:35.373321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa8ec6-98eb-4281-97cb-fd38530f7b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:35.383184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa8ec6-98eb-4281-97cb-fd38530f7b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:35.384644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa8ec6-98eb-4281-97cb-fd38530f7b7f service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:35.394034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 771/1533] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:38.885700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9e938-039c-4d44-be9c-88b25eb0be05 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:38.886269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9e938-039c-4d44-be9c-88b25eb0be05 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:38.886269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9e938-039c-4d44-be9c-88b25eb0be05 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:38.887611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:18:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:38.892632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358f0b6-c9f2-4402-b0b4-19ae9b8d407d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:38.893032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358f0b6-c9f2-4402-b0b4-19ae9b8d407d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:38.893183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358f0b6-c9f2-4402-b0b4-19ae9b8d407d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:38.913580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4358f0b6-c9f2-4402-b0b4-19ae9b8d407d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:38.914781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4358f0b6-c9f2-4402-b0b4-19ae9b8d407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:38.922654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:18:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:39.228463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9e938-039c-4d44-be9c-88b25eb0be05 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:39.230160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-29f9e938-039c-4d44-be9c-88b25eb0be05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:39.295943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:18:39.310963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9e938-039c-4d44-be9c-88b25eb0be05 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:39.314546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d479859-efbf-46ff-a116-d8851f7ecbf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:39.315166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479859-efbf-46ff-a116-d8851f7ecbf7 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:39.315446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479859-efbf-46ff-a116-d8851f7ecbf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:39.315718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479859-efbf-46ff-a116-d8851f7ecbf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:39.325337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d479859-efbf-46ff-a116-d8851f7ecbf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:39.326856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d479859-efbf-46ff-a116-d8851f7ecbf7 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:39.336493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 764/1536] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:41.348788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa325b28-d27c-4c06-b4bf-0cfdac451ae0 None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:41.349341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa325b28-d27c-4c06-b4bf-0cfdac451ae0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:41.349341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa325b28-d27c-4c06-b4bf-0cfdac451ae0 None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:41.350706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 22:18:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:18:41.356063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a93ab-b3dc-4afa-b452-d61cfd11a6ad None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:41.356503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a93ab-b3dc-4afa-b452-d61cfd11a6ad None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:41.356690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a93ab-b3dc-4afa-b452-d61cfd11a6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:41.377327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-da5a93ab-b3dc-4afa-b452-d61cfd11a6ad None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:18:41.378587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-da5a93ab-b3dc-4afa-b452-d61cfd11a6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:41.386692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 22:18:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:18:41.689011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa325b28-d27c-4c06-b4bf-0cfdac451ae0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:18:41.690707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fa325b28-d27c-4c06-b4bf-0cfdac451ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:18:41.704105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa325b28-d27c-4c06-b4bf-0cfdac451ae0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:18:41.708320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b31947-4a20-4ad9-9918-1f8e8d6da9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:18:41.708929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31947-4a20-4ad9-9918-1f8e8d6da9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:18:41.709208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31947-4a20-4ad9-9918-1f8e8d6da9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:18:41.709533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31947-4a20-4ad9-9918-1f8e8d6da9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:18:41.718419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b31947-4a20-4ad9-9918-1f8e8d6da9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:18:41.719843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b31947-4a20-4ad9-9918-1f8e8d6da9d5 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:18:41.729976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 774/1539] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:18:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:18:51.371499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=11 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:18:51.725869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:05.715642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b1cf6-700b-4d44-9a2a-4c3028f9b8ba None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:05.716327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b1cf6-700b-4d44-9a2a-4c3028f9b8ba None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:05.716327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b1cf6-700b-4d44-9a2a-4c3028f9b8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:05.742932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6a2b1cf6-700b-4d44-9a2a-4c3028f9b8ba None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:05.744209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6a2b1cf6-700b-4d44-9a2a-4c3028f9b8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:05.757192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 766/1540] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:05.766483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469185-5765-4650-8536-3fb7adc32f6b None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:05.767102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469185-5765-4650-8536-3fb7adc32f6b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:05.767102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469185-5765-4650-8536-3fb7adc32f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:05.792435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3b469185-5765-4650-8536-3fb7adc32f6b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:05.798452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3b469185-5765-4650-8536-3fb7adc32f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:05.808404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 775/1541] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:05.812920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b1cf6-700b-4d44-9a2a-4c3028f9b8ba None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:05.828040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b1cf6-700b-4d44-9a2a-4c3028f9b8ba None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:05.837870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b5297-255b-4c64-b3e3-63e2034a9216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:05.838904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5297-255b-4c64-b3e3-63e2034a9216 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:05.839392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5297-255b-4c64-b3e3-63e2034a9216 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:05.839865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5297-255b-4c64-b3e3-63e2034a9216 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:05.840969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b5297-255b-4c64-b3e3-63e2034a9216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:05.841507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b5297-255b-4c64-b3e3-63e2034a9216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:05.843778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b5297-255b-4c64-b3e3-63e2034a9216 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:05.868256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 767/1542] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:05.877504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469185-5765-4650-8536-3fb7adc32f6b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:05.892517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469185-5765-4650-8536-3fb7adc32f6b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:05.902231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82d70c-0cec-431e-a86f-776e539323b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:05.902881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82d70c-0cec-431e-a86f-776e539323b9 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:05.903160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82d70c-0cec-431e-a86f-776e539323b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:05.903432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82d70c-0cec-431e-a86f-776e539323b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:05.904173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82d70c-0cec-431e-a86f-776e539323b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:05.906128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82d70c-0cec-431e-a86f-776e539323b9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:05.929827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff2a4-60ed-4503-8fd0-2ca1b05f529d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:05.930441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff2a4-60ed-4503-8fd0-2ca1b05f529d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:05.930922 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff2a4-60ed-4503-8fd0-2ca1b05f529d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:05.963637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-663ff2a4-60ed-4503-8fd0-2ca1b05f529d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:05.965213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-663ff2a4-60ed-4503-8fd0-2ca1b05f529d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:05.973501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 776/1543] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:05 2022] POST /identity/v3/projects => generated 390 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:05.973501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82d70c-0cec-431e-a86f-776e539323b9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.093365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82d70c-0cec-431e-a86f-776e539323b9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.113178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00976b-45aa-4c9c-a692-dfb5bd0d2efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.113855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00976b-45aa-4c9c-a692-dfb5bd0d2efc admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.114320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00976b-45aa-4c9c-a692-dfb5bd0d2efc admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.114878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00976b-45aa-4c9c-a692-dfb5bd0d2efc admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.115795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00976b-45aa-4c9c-a692-dfb5bd0d2efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.116990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WARNING py.warnings [None req-1a00976b-45aa-4c9c-a692-dfb5bd0d2efc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:20:06.116990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: warnings.warn(msg) Aug 04 22:20:06.116990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: Aug 04 22:20:06.118651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00976b-45aa-4c9c-a692-dfb5bd0d2efc admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.122544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 768/1544] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:06.124087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b85f-a5b9-44b4-8817-caec77a8c7d3 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.124601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b85f-a5b9-44b4-8817-caec77a8c7d3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.125236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b85f-a5b9-44b4-8817-caec77a8c7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.148844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 777/1545] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:05 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:20:06.150277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0973e-bccb-4848-bc3c-07f02b20ee10 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.150571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0973e-bccb-4848-bc3c-07f02b20ee10 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.150902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0973e-bccb-4848-bc3c-07f02b20ee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.151876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ea51b85f-a5b9-44b4-8817-caec77a8c7d3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.153395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ea51b85f-a5b9-44b4-8817-caec77a8c7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.166878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 769/1546] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.168544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd2a4-2479-49a9-857b-428b2988cd07 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.169067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd2a4-2479-49a9-857b-428b2988cd07 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.169599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd2a4-2479-49a9-857b-428b2988cd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.173793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a1a0973e-bccb-4848-bc3c-07f02b20ee10 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.175216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a1a0973e-bccb-4848-bc3c-07f02b20ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.187424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 778/1547] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.188983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d43e70-7393-4c31-bc7d-a5e1c5ff078a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.189273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d43e70-7393-4c31-bc7d-a5e1c5ff078a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.189575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d43e70-7393-4c31-bc7d-a5e1c5ff078a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.190478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-780fd2a4-2479-49a9-857b-428b2988cd07 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.191686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-780fd2a4-2479-49a9-857b-428b2988cd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.200905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 770/1548] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.202174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e37e0-a774-47df-9620-9a6381ccc61b None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.202463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e37e0-a774-47df-9620-9a6381ccc61b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.202736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e37e0-a774-47df-9620-9a6381ccc61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.212448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e6d43e70-7393-4c31-bc7d-a5e1c5ff078a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.213943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e6d43e70-7393-4c31-bc7d-a5e1c5ff078a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.222105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 779/1549] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.222903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d43e70-7393-4c31-bc7d-a5e1c5ff078a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.224803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9f3e37e0-a774-47df-9620-9a6381ccc61b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.225987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f3e37e0-a774-47df-9620-9a6381ccc61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.233636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d43e70-7393-4c31-bc7d-a5e1c5ff078a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.234503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 771/1550] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.235841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb1ea-b96b-4ee3-9111-64c520312c4c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.236124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb1ea-b96b-4ee3-9111-64c520312c4c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.236397 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb1ea-b96b-4ee3-9111-64c520312c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.238638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627de6cc-f5e6-4aa7-ace4-8461a15ecb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.239241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627de6cc-f5e6-4aa7-ace4-8461a15ecb5a admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.239525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627de6cc-f5e6-4aa7-ace4-8461a15ecb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.239886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627de6cc-f5e6-4aa7-ace4-8461a15ecb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.241089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627de6cc-f5e6-4aa7-ace4-8461a15ecb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.242712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627de6cc-f5e6-4aa7-ace4-8461a15ecb5a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.253305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 780/1551] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:06.254427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267319-4408-454e-ac17-134eeacbb701 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.254693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267319-4408-454e-ac17-134eeacbb701 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.254970 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267319-4408-454e-ac17-134eeacbb701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.256465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-233bb1ea-b96b-4ee3-9111-64c520312c4c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.257898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-233bb1ea-b96b-4ee3-9111-64c520312c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.265866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 772/1552] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.267053 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69321-25c6-45e5-af83-c5b5ccf0bcd7 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.267337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69321-25c6-45e5-af83-c5b5ccf0bcd7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.267614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69321-25c6-45e5-af83-c5b5ccf0bcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.277248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-18267319-4408-454e-ac17-134eeacbb701 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.278478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-18267319-4408-454e-ac17-134eeacbb701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.287275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 781/1553] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.287457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-87a69321-25c6-45e5-af83-c5b5ccf0bcd7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.288439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16848103-3974-4b48-8697-978e1abf78ed None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.288637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-87a69321-25c6-45e5-af83-c5b5ccf0bcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.288760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16848103-3974-4b48-8697-978e1abf78ed None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.288977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16848103-3974-4b48-8697-978e1abf78ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.296582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 773/1554] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.297809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f4316-9264-4399-b01a-a019004c2f35 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.298098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f4316-9264-4399-b01a-a019004c2f35 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.298376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f4316-9264-4399-b01a-a019004c2f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.308779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-16848103-3974-4b48-8697-978e1abf78ed None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.310001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-16848103-3974-4b48-8697-978e1abf78ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.317131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b52f4316-9264-4399-b01a-a019004c2f35 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.318390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 782/1555] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.318857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b52f4316-9264-4399-b01a-a019004c2f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.320172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b432eb9-481b-4a5d-a238-44f1bad47ce9 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.320458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b432eb9-481b-4a5d-a238-44f1bad47ce9 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.320731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b432eb9-481b-4a5d-a238-44f1bad47ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.328527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 774/1556] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.329317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f4316-9264-4399-b01a-a019004c2f35 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.339573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f4316-9264-4399-b01a-a019004c2f35 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.340988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8b432eb9-481b-4a5d-a238-44f1bad47ce9 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.342206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8b432eb9-481b-4a5d-a238-44f1bad47ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.344226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f2dd7-1cea-45c6-8012-9f30dd3b5d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.344875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f2dd7-1cea-45c6-8012-9f30dd3b5d6f admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.345142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f2dd7-1cea-45c6-8012-9f30dd3b5d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.345432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f2dd7-1cea-45c6-8012-9f30dd3b5d6f admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.346130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f2dd7-1cea-45c6-8012-9f30dd3b5d6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:06.346433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f2dd7-1cea-45c6-8012-9f30dd3b5d6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.348041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f2dd7-1cea-45c6-8012-9f30dd3b5d6f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.350429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 783/1557] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.351299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b432eb9-481b-4a5d-a238-44f1bad47ce9 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.365682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 775/1558] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:06.365896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b432eb9-481b-4a5d-a238-44f1bad47ce9 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.366002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156882-7209-431e-a44d-71d8c91dac35 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.366002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156882-7209-431e-a44d-71d8c91dac35 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.366002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156882-7209-431e-a44d-71d8c91dac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.369922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa4223-5930-4b19-996a-0b09bdc14ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.370534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa4223-5930-4b19-996a-0b09bdc14ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.370868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa4223-5930-4b19-996a-0b09bdc14ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.371176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa4223-5930-4b19-996a-0b09bdc14ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.371866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa4223-5930-4b19-996a-0b09bdc14ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.373569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa4223-5930-4b19-996a-0b09bdc14ad9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.382973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-35156882-7209-431e-a44d-71d8c91dac35 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.384187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-35156882-7209-431e-a44d-71d8c91dac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.387876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 784/1559] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:06.387876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa4223-5930-4b19-996a-0b09bdc14ad9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.395625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 776/1560] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.395625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b1494-3bca-4c76-9c11-1c6f2ef04392 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.395625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b1494-3bca-4c76-9c11-1c6f2ef04392 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.395625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b1494-3bca-4c76-9c11-1c6f2ef04392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.396062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa4223-5930-4b19-996a-0b09bdc14ad9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.400517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.401113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.401443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.401728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.402433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:06.402735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.403795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WARNING py.warnings [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:20:06.403795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: warnings.warn(msg) Aug 04 22:20:06.403795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: Aug 04 22:20:06.404664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.414521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-125b1494-3bca-4c76-9c11-1c6f2ef04392 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.415736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-125b1494-3bca-4c76-9c11-1c6f2ef04392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.417957 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 785/1561] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:06.418829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.423931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 777/1562] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.424605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b1494-3bca-4c76-9c11-1c6f2ef04392 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.431619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf66959-1679-4155-8919-37617bf0dad5 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.434607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b1494-3bca-4c76-9c11-1c6f2ef04392 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.436309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.436948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.437228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.437531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.438224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:06.438526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.439502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.440099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.440197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.440379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.440656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.441348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:06.441757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.443350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.452473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 786/1563] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:06.453105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.462381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 778/1564] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:06.462381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.464899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e11791-9be3-44c6-a634-e03009660873 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.469615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.470261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.470539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.470814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.471506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:06.471810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.471943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241637d-42eb-4a98-b19c-88651cd7b8d6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.473440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.478114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.478668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.478951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.479226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.479917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.481006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING py.warnings [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:20:06.481006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: warnings.warn(msg) Aug 04 22:20:06.481006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: Aug 04 22:20:06.482771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.486321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 787/1565] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:06.487101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.497637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ba78b-cc23-47db-84f1-c8fcb7480505 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.502533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db867d-d3b8-4c0b-9b49-4d2eab581629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.503191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db867d-d3b8-4c0b-9b49-4d2eab581629 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.503497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db867d-d3b8-4c0b-9b49-4d2eab581629 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.503980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db867d-d3b8-4c0b-9b49-4d2eab581629 admin admin] PATH_INFO: `/v3/projects/21d013d03bbc4f8ead46aca55d441eb1/users/8c0aa2b9a5ac46f9bede8d007dcd3504/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.515609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 779/1566] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/projects => generated 386 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:06.516963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.520436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db867d-d3b8-4c0b-9b49-4d2eab581629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d013d03bbc4f8ead46aca55d441eb1, user_id=8c0aa2b9a5ac46f9bede8d007dcd3504, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.523165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db867d-d3b8-4c0b-9b49-4d2eab581629 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.525478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:06.539301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 788/1567] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:06 2022] PUT /identity/v3/projects/21d013d03bbc4f8ead46aca55d441eb1/users/8c0aa2b9a5ac46f9bede8d007dcd3504/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:06.540173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db867d-d3b8-4c0b-9b49-4d2eab581629 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.568046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:06.589493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d313e37-f890-476d-9b08-0a0baa553532 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.608040 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db867d-d3b8-4c0b-9b49-4d2eab581629 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.612741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda5eeb-6b1e-43c6-8e86-05c1f5b8036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.613352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5eeb-6b1e-43c6-8e86-05c1f5b8036e admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.613672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5eeb-6b1e-43c6-8e86-05c1f5b8036e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.614019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5eeb-6b1e-43c6-8e86-05c1f5b8036e admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.614668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda5eeb-6b1e-43c6-8e86-05c1f5b8036e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.617237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda5eeb-6b1e-43c6-8e86-05c1f5b8036e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.645913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 789/1568] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/projects => generated 366 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:06.647328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda5eeb-6b1e-43c6-8e86-05c1f5b8036e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.651799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.652439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.652716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.652993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.653736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:06.654036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.655656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.668518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 780/1569] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:06.669325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.679618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf4d66-1689-4e69-85a2-ecc8ea3ba22b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.715644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda5eeb-6b1e-43c6-8e86-05c1f5b8036e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.720327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.720969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.721244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.721552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.722249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:06.722546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.725357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.736942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 790/1570] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:06.737691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.737993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf48fd-f76a-4adf-95b1-51f15ba31a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.738621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf48fd-f76a-4adf-95b1-51f15ba31a19 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.738899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf48fd-f76a-4adf-95b1-51f15ba31a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.739169 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf48fd-f76a-4adf-95b1-51f15ba31a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.739874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf48fd-f76a-4adf-95b1-51f15ba31a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.741926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf48fd-f76a-4adf-95b1-51f15ba31a19 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.753818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2778bb-2b32-41aa-b0fc-4ab8f9749eb3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.759299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f6afc-d86b-40a4-9a4b-b50d42b0cef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.760003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f6afc-d86b-40a4-9a4b-b50d42b0cef6 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.760279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f6afc-d86b-40a4-9a4b-b50d42b0cef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.760624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f6afc-d86b-40a4-9a4b-b50d42b0cef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.761437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f6afc-d86b-40a4-9a4b-b50d42b0cef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.763323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f6afc-d86b-40a4-9a4b-b50d42b0cef6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.772953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 781/1571] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/projects => generated 374 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:06.772953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf48fd-f76a-4adf-95b1-51f15ba31a19 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.796855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 791/1572] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/projects => generated 394 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:06.796855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f6afc-d86b-40a4-9a4b-b50d42b0cef6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.835275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf48fd-f76a-4adf-95b1-51f15ba31a19 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.845517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:06.852249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:06.865212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f6afc-d86b-40a4-9a4b-b50d42b0cef6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.870332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462eff6a-1ebb-4c51-9355-442b1f89e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.870954 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eff6a-1ebb-4c51-9355-442b1f89e2be admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.871234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eff6a-1ebb-4c51-9355-442b1f89e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.871505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eff6a-1ebb-4c51-9355-442b1f89e2be admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.872174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462eff6a-1ebb-4c51-9355-442b1f89e2be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.874237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462eff6a-1ebb-4c51-9355-442b1f89e2be admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.896430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 792/1573] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:06.896430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f237f-0f04-4911-bdba-7f9093d072b6 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.896430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f237f-0f04-4911-bdba-7f9093d072b6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.896430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f237f-0f04-4911-bdba-7f9093d072b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.899885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f86733-2292-49e4-97b3-c01bc786017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.899885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f86733-2292-49e4-97b3-c01bc786017d admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.899885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f86733-2292-49e4-97b3-c01bc786017d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.899885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f86733-2292-49e4-97b3-c01bc786017d admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:06.900536 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f86733-2292-49e4-97b3-c01bc786017d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:06.902551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f86733-2292-49e4-97b3-c01bc786017d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:06.918017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e56f237f-0f04-4911-bdba-7f9093d072b6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:06.919415 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e56f237f-0f04-4911-bdba-7f9093d072b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:06.928085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 793/1574] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:06.929059 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f237f-0f04-4911-bdba-7f9093d072b6 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.935118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 782/1575] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/projects => generated 384 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:06.936114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f86733-2292-49e4-97b3-c01bc786017d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:06.943032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f237f-0f04-4911-bdba-7f9093d072b6 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.997183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f86733-2292-49e4-97b3-c01bc786017d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:06.998204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217d60c-05ff-4b2f-a566-c98da46fa021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:06.998822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d60c-05ff-4b2f-a566-c98da46fa021 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:06.999092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d60c-05ff-4b2f-a566-c98da46fa021 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:06.999356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d60c-05ff-4b2f-a566-c98da46fa021 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.000031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217d60c-05ff-4b2f-a566-c98da46fa021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.002186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217d60c-05ff-4b2f-a566-c98da46fa021 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.002321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb5c2-cb0f-4b75-a456-c19773edbbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.002874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb5c2-cb0f-4b75-a456-c19773edbbdc admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.003157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb5c2-cb0f-4b75-a456-c19773edbbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.003432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb5c2-cb0f-4b75-a456-c19773edbbdc admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.004100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cb5c2-cb0f-4b75-a456-c19773edbbdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.006589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cb5c2-cb0f-4b75-a456-c19773edbbdc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.024490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 794/1576] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:07.025518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217d60c-05ff-4b2f-a566-c98da46fa021 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.035582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217d60c-05ff-4b2f-a566-c98da46fa021 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.041802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 783/1577] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:06 2022] POST /identity/v3/projects => generated 374 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:07.041802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb5c2-cb0f-4b75-a456-c19773edbbdc admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.043748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff428b2-9048-45a1-8330-68b23dab0639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.044389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff428b2-9048-45a1-8330-68b23dab0639 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.044656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff428b2-9048-45a1-8330-68b23dab0639 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.044921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff428b2-9048-45a1-8330-68b23dab0639 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.045604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff428b2-9048-45a1-8330-68b23dab0639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.047495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff428b2-9048-45a1-8330-68b23dab0639 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.078796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 795/1578] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:07 2022] POST /identity/v3/projects => generated 380 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:07.078796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff428b2-9048-45a1-8330-68b23dab0639 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.104073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb5c2-cb0f-4b75-a456-c19773edbbdc admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.138417 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:07.141207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff428b2-9048-45a1-8330-68b23dab0639 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.146423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197ecdb-25bb-479d-952c-2bdae2db3f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.147062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197ecdb-25bb-479d-952c-2bdae2db3f17 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.147333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197ecdb-25bb-479d-952c-2bdae2db3f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.147609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197ecdb-25bb-479d-952c-2bdae2db3f17 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.148290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197ecdb-25bb-479d-952c-2bdae2db3f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.150249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197ecdb-25bb-479d-952c-2bdae2db3f17 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.168334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5fab-0bb7-4b65-a4d1-0f47c0f2bd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.168992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5fab-0bb7-4b65-a4d1-0f47c0f2bd1b admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.169273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5fab-0bb7-4b65-a4d1-0f47c0f2bd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.169579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5fab-0bb7-4b65-a4d1-0f47c0f2bd1b admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.170319 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b5fab-0bb7-4b65-a4d1-0f47c0f2bd1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.172224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b5fab-0bb7-4b65-a4d1-0f47c0f2bd1b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.172668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 796/1579] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:07 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:07.173517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197ecdb-25bb-479d-952c-2bdae2db3f17 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.185047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197ecdb-25bb-479d-952c-2bdae2db3f17 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.190178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995b9b-24c7-4dda-879e-be3733324f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.190788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995b9b-24c7-4dda-879e-be3733324f62 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.191069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995b9b-24c7-4dda-879e-be3733324f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.191344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995b9b-24c7-4dda-879e-be3733324f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.192032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20995b9b-24c7-4dda-879e-be3733324f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.193679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 784/1580] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:07 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:07.193977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20995b9b-24c7-4dda-879e-be3733324f62 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.194508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5fab-0bb7-4b65-a4d1-0f47c0f2bd1b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.205044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 797/1581] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.205751 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995b9b-24c7-4dda-879e-be3733324f62 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.215715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995b9b-24c7-4dda-879e-be3733324f62 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.220396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ec9bf-fb36-4486-8c8b-03d49a36b757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.220995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ec9bf-fb36-4486-8c8b-03d49a36b757 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.221285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ec9bf-fb36-4486-8c8b-03d49a36b757 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.221590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ec9bf-fb36-4486-8c8b-03d49a36b757 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.222244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ec9bf-fb36-4486-8c8b-03d49a36b757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.224820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ec9bf-fb36-4486-8c8b-03d49a36b757 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.247461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 798/1582] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:07 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:07.248209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ec9bf-fb36-4486-8c8b-03d49a36b757 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.254747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5fab-0bb7-4b65-a4d1-0f47c0f2bd1b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.255402 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:07.258790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcd520-5457-456e-96f0-c4fb6d36c815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.259324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd520-5457-456e-96f0-c4fb6d36c815 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.259645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd520-5457-456e-96f0-c4fb6d36c815 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.260071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd520-5457-456e-96f0-c4fb6d36c815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.262410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ec9bf-fb36-4486-8c8b-03d49a36b757 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.268239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524678f-e5da-474c-a5dd-5d284607004a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.268926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524678f-e5da-474c-a5dd-5d284607004a admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.269057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcd520-5457-456e-96f0-c4fb6d36c815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.269343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524678f-e5da-474c-a5dd-5d284607004a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.269725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524678f-e5da-474c-a5dd-5d284607004a admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.270491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524678f-e5da-474c-a5dd-5d284607004a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.270588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcd520-5457-456e-96f0-c4fb6d36c815 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.272211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524678f-e5da-474c-a5dd-5d284607004a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.280091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 785/1583] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:07.280932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcd520-5457-456e-96f0-c4fb6d36c815 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.285362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 799/1584] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.286240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524678f-e5da-474c-a5dd-5d284607004a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.291715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcd520-5457-456e-96f0-c4fb6d36c815 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.296748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebebbf6-86f2-417a-abd2-3bd42e7900fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.297352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebebbf6-86f2-417a-abd2-3bd42e7900fa admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.297512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524678f-e5da-474c-a5dd-5d284607004a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.297712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebebbf6-86f2-417a-abd2-3bd42e7900fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.297945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebebbf6-86f2-417a-abd2-3bd42e7900fa admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.298604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebebbf6-86f2-417a-abd2-3bd42e7900fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.300815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebebbf6-86f2-417a-abd2-3bd42e7900fa admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.304421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7674d-408f-492b-a06f-3bf4aa17dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.305330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7674d-408f-492b-a06f-3bf4aa17dcb5 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.305688 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7674d-408f-492b-a06f-3bf4aa17dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.305964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7674d-408f-492b-a06f-3bf4aa17dcb5 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.306635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7674d-408f-492b-a06f-3bf4aa17dcb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.308756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7674d-408f-492b-a06f-3bf4aa17dcb5 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.326142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 786/1585] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:07 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:07.327119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebebbf6-86f2-417a-abd2-3bd42e7900fa admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.333036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 800/1586] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:07 2022] POST /identity/v3/users => generated 291 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:07.334522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7674d-408f-492b-a06f-3bf4aa17dcb5 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.339615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebebbf6-86f2-417a-abd2-3bd42e7900fa admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.346091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4019dc6-b941-45c9-a374-4398ce13259a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.346322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7674d-408f-492b-a06f-3bf4aa17dcb5 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.346812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4019dc6-b941-45c9-a374-4398ce13259a admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.347148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4019dc6-b941-45c9-a374-4398ce13259a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.347548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4019dc6-b941-45c9-a374-4398ce13259a admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.348328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4019dc6-b941-45c9-a374-4398ce13259a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.350666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4019dc6-b941-45c9-a374-4398ce13259a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.352602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26cf82-9058-4859-af67-d1dec6469272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.353447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26cf82-9058-4859-af67-d1dec6469272 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.353898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26cf82-9058-4859-af67-d1dec6469272 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.354327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26cf82-9058-4859-af67-d1dec6469272 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.355205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26cf82-9058-4859-af67-d1dec6469272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.356963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26cf82-9058-4859-af67-d1dec6469272 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.365700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 787/1587] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.367000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4019dc6-b941-45c9-a374-4398ce13259a admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.376746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 801/1588] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.377699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26cf82-9058-4859-af67-d1dec6469272 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.378189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4019dc6-b941-45c9-a374-4398ce13259a admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.384730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80004136-b53a-4bd1-9e1f-8feba6a49e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.386077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80004136-b53a-4bd1-9e1f-8feba6a49e2a admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.386395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80004136-b53a-4bd1-9e1f-8feba6a49e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.386666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80004136-b53a-4bd1-9e1f-8feba6a49e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.387392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80004136-b53a-4bd1-9e1f-8feba6a49e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.389056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80004136-b53a-4bd1-9e1f-8feba6a49e2a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.389440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26cf82-9058-4859-af67-d1dec6469272 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.394283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e689f3e-4c37-484c-a945-791540cf2580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.395121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e689f3e-4c37-484c-a945-791540cf2580 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.395279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e689f3e-4c37-484c-a945-791540cf2580 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.395477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e689f3e-4c37-484c-a945-791540cf2580 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.396714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e689f3e-4c37-484c-a945-791540cf2580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.398348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e689f3e-4c37-484c-a945-791540cf2580 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.404750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 788/1589] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.404750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80004136-b53a-4bd1-9e1f-8feba6a49e2a admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.413675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 802/1590] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.413675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e689f3e-4c37-484c-a945-791540cf2580 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.418802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80004136-b53a-4bd1-9e1f-8feba6a49e2a admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.422697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e689f3e-4c37-484c-a945-791540cf2580 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.423910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f33b9-3e39-4b79-9fc4-64a554a4e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.424638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f33b9-3e39-4b79-9fc4-64a554a4e4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.425031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f33b9-3e39-4b79-9fc4-64a554a4e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.425379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f33b9-3e39-4b79-9fc4-64a554a4e4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.426568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f33b9-3e39-4b79-9fc4-64a554a4e4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.428595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f33b9-3e39-4b79-9fc4-64a554a4e4c6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.429453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3702ac-67e6-4f30-80b0-4880620a08d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.430832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3702ac-67e6-4f30-80b0-4880620a08d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.430832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3702ac-67e6-4f30-80b0-4880620a08d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.430832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3702ac-67e6-4f30-80b0-4880620a08d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.431706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3702ac-67e6-4f30-80b0-4880620a08d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.433495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3702ac-67e6-4f30-80b0-4880620a08d3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.442271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 789/1591] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.449440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 803/1592] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.449629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f33b9-3e39-4b79-9fc4-64a554a4e4c6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.449731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3702ac-67e6-4f30-80b0-4880620a08d3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.459952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f33b9-3e39-4b79-9fc4-64a554a4e4c6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.460620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3702ac-67e6-4f30-80b0-4880620a08d3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.465214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf3117-37dc-48d2-8812-001bfecb1e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.465887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3117-37dc-48d2-8812-001bfecb1e74 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.466132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158d91a-15d1-46e3-8ad3-1b63074f44d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.466269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3117-37dc-48d2-8812-001bfecb1e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.466428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3117-37dc-48d2-8812-001bfecb1e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.467000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158d91a-15d1-46e3-8ad3-1b63074f44d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.467123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf3117-37dc-48d2-8812-001bfecb1e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.467461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158d91a-15d1-46e3-8ad3-1b63074f44d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.467821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158d91a-15d1-46e3-8ad3-1b63074f44d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.468672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158d91a-15d1-46e3-8ad3-1b63074f44d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.468789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf3117-37dc-48d2-8812-001bfecb1e74 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.471090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158d91a-15d1-46e3-8ad3-1b63074f44d7 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.479505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 790/1593] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.479505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf3117-37dc-48d2-8812-001bfecb1e74 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.484168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 804/1594] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.485237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158d91a-15d1-46e3-8ad3-1b63074f44d7 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.502578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158d91a-15d1-46e3-8ad3-1b63074f44d7 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.511151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf3117-37dc-48d2-8812-001bfecb1e74 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.511666 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629846e-c2e2-4ca3-aa8c-1b4874b36dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.512718 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629846e-c2e2-4ca3-aa8c-1b4874b36dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.513073 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629846e-c2e2-4ca3-aa8c-1b4874b36dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.513572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629846e-c2e2-4ca3-aa8c-1b4874b36dd3 admin admin] PATH_INFO: `/v3/projects/7f5901777ea64cf995c5626bcc782115/users/bb940b4935e54eeb9409fb02b05c5e02/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.516613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505dfb-41e3-4210-b1fb-74d30a249e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.517236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505dfb-41e3-4210-b1fb-74d30a249e01 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.517565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505dfb-41e3-4210-b1fb-74d30a249e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.517844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505dfb-41e3-4210-b1fb-74d30a249e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.518541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea505dfb-41e3-4210-b1fb-74d30a249e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.520166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea505dfb-41e3-4210-b1fb-74d30a249e01 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.530939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629846e-c2e2-4ca3-aa8c-1b4874b36dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5901777ea64cf995c5626bcc782115, user_id=bb940b4935e54eeb9409fb02b05c5e02, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.533657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 791/1595] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.533917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629846e-c2e2-4ca3-aa8c-1b4874b36dd3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.535031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505dfb-41e3-4210-b1fb-74d30a249e01 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.546431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505dfb-41e3-4210-b1fb-74d30a249e01 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.549946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 805/1596] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:07 2022] PUT /identity/v3/projects/7f5901777ea64cf995c5626bcc782115/users/bb940b4935e54eeb9409fb02b05c5e02/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:07.550687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629846e-c2e2-4ca3-aa8c-1b4874b36dd3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.616587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:07.629050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629846e-c2e2-4ca3-aa8c-1b4874b36dd3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.629483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30404a61-78cc-4226-9c2a-879a72b72420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.630074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404a61-78cc-4226-9c2a-879a72b72420 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.631481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404a61-78cc-4226-9c2a-879a72b72420 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.631803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404a61-78cc-4226-9c2a-879a72b72420 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.632527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30404a61-78cc-4226-9c2a-879a72b72420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.634295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30404a61-78cc-4226-9c2a-879a72b72420 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.635619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38107ca1-e17d-41e2-8926-7d03f689493c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.636813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38107ca1-e17d-41e2-8926-7d03f689493c admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.637106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38107ca1-e17d-41e2-8926-7d03f689493c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.637567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38107ca1-e17d-41e2-8926-7d03f689493c admin admin] PATH_INFO: `/v3/projects/92c96f8280d94bf4b15c7fdbad9b17ac/users/337832e987334f5d8def83bca7331497/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.651079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 792/1597] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.651079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30404a61-78cc-4226-9c2a-879a72b72420 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.660049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38107ca1-e17d-41e2-8926-7d03f689493c admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c96f8280d94bf4b15c7fdbad9b17ac, user_id=337832e987334f5d8def83bca7331497, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.663406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38107ca1-e17d-41e2-8926-7d03f689493c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.675091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30404a61-78cc-4226-9c2a-879a72b72420 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.690314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 806/1598] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:07 2022] PUT /identity/v3/projects/92c96f8280d94bf4b15c7fdbad9b17ac/users/337832e987334f5d8def83bca7331497/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:07.690314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38107ca1-e17d-41e2-8926-7d03f689493c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.748052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:07.762260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30129b8d-3eb1-4332-829a-42777a75755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.762926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30129b8d-3eb1-4332-829a-42777a75755e admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.763194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30129b8d-3eb1-4332-829a-42777a75755e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.763444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30129b8d-3eb1-4332-829a-42777a75755e admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.764148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30129b8d-3eb1-4332-829a-42777a75755e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.765802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30129b8d-3eb1-4332-829a-42777a75755e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.766053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38107ca1-e17d-41e2-8926-7d03f689493c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.772337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f0a9-14fe-41cc-afb4-e56d2aad3565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.773247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f0a9-14fe-41cc-afb4-e56d2aad3565 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.773678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f0a9-14fe-41cc-afb4-e56d2aad3565 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.774229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f0a9-14fe-41cc-afb4-e56d2aad3565 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.774937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9f0a9-14fe-41cc-afb4-e56d2aad3565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.776549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9f0a9-14fe-41cc-afb4-e56d2aad3565 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.780659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 793/1599] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.780659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30129b8d-3eb1-4332-829a-42777a75755e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.788872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 807/1600] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:07 2022] GET /identity/v3/roles => generated 1427 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:07.789725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f0a9-14fe-41cc-afb4-e56d2aad3565 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.794972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30129b8d-3eb1-4332-829a-42777a75755e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.799824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f0a9-14fe-41cc-afb4-e56d2aad3565 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.800827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa57ced-6bd3-481a-906c-5e52ed97e622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.802024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa57ced-6bd3-481a-906c-5e52ed97e622 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.803097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa57ced-6bd3-481a-906c-5e52ed97e622 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.803953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa57ced-6bd3-481a-906c-5e52ed97e622 admin admin] PATH_INFO: `/v3/projects/5116986e2edc4b5c92da0b77a497910a/users/f3dab07011da408c97ed90d58f2c155b/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.806329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339c768-e3d5-4fb7-b313-9b163d2e8b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.806928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339c768-e3d5-4fb7-b313-9b163d2e8b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.807213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339c768-e3d5-4fb7-b313-9b163d2e8b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.807484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339c768-e3d5-4fb7-b313-9b163d2e8b36 admin admin] PATH_INFO: `/v3/projects/c940df02ebff4e94bf0874df846db424/users/e6066712c4604dcfb7407d92230d33d3/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.821458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa57ced-6bd3-481a-906c-5e52ed97e622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5116986e2edc4b5c92da0b77a497910a, user_id=f3dab07011da408c97ed90d58f2c155b, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.822561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING py.warnings [None req-9aa57ced-6bd3-481a-906c-5e52ed97e622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:20:07.822561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: warnings.warn(msg) Aug 04 22:20:07.822561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: Aug 04 22:20:07.825277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa57ced-6bd3-481a-906c-5e52ed97e622 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.825665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339c768-e3d5-4fb7-b313-9b163d2e8b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c940df02ebff4e94bf0874df846db424, user_id=e6066712c4604dcfb7407d92230d33d3, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.828514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339c768-e3d5-4fb7-b313-9b163d2e8b36 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.841976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 794/1601] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:07 2022] PUT /identity/v3/projects/5116986e2edc4b5c92da0b77a497910a/users/f3dab07011da408c97ed90d58f2c155b/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:07.843466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54a8a1-f9de-432a-bbe3-183a2b9fd774 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.843823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54a8a1-f9de-432a-bbe3-183a2b9fd774 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.844174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54a8a1-f9de-432a-bbe3-183a2b9fd774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.845708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 808/1602] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:07 2022] PUT /identity/v3/projects/c940df02ebff4e94bf0874df846db424/users/e6066712c4604dcfb7407d92230d33d3/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:07.846984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339c768-e3d5-4fb7-b313-9b163d2e8b36 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:07.882768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-dc54a8a1-f9de-432a-bbe3-183a2b9fd774 None None] MFA Rules not processed for user `bb940b4935e54eeb9409fb02b05c5e02`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:07.884014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-dc54a8a1-f9de-432a-bbe3-183a2b9fd774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:07.932218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339c768-e3d5-4fb7-b313-9b163d2e8b36 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:07.942921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda84c0-be53-4c7b-95eb-cbbf889f47ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:07.943279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda84c0-be53-4c7b-95eb-cbbf889f47ce admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.943836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda84c0-be53-4c7b-95eb-cbbf889f47ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.944325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda84c0-be53-4c7b-95eb-cbbf889f47ce admin admin] PATH_INFO: `/v3/projects/011b296da6654fa38f76eadd7bebae6f/users/8c50160e51b54cde9a3f617d71fbf1c0/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.961551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda84c0-be53-4c7b-95eb-cbbf889f47ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=011b296da6654fa38f76eadd7bebae6f, user_id=8c50160e51b54cde9a3f617d71fbf1c0, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:07.964281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda84c0-be53-4c7b-95eb-cbbf889f47ce admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:07.971688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 795/1603] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:07.971688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c7c5d-1e2b-4334-b8ff-ab71a7b6df14 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:07.971688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c7c5d-1e2b-4334-b8ff-ab71a7b6df14 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:07.971688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c7c5d-1e2b-4334-b8ff-ab71a7b6df14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:07.995862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 809/1604] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:07 2022] PUT /identity/v3/projects/011b296da6654fa38f76eadd7bebae6f/users/8c50160e51b54cde9a3f617d71fbf1c0/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:07.995862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda84c0-be53-4c7b-95eb-cbbf889f47ce admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.005927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-416c7c5d-1e2b-4334-b8ff-ab71a7b6df14 None None] MFA Rules not processed for user `337832e987334f5d8def83bca7331497`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:08.007496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-416c7c5d-1e2b-4334-b8ff-ab71a7b6df14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:08.072035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:08.097883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda84c0-be53-4c7b-95eb-cbbf889f47ce admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:08.112189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402db29b-30f6-4dcd-97d0-bcbf0222d877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:08.113025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db29b-30f6-4dcd-97d0-bcbf0222d877 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.115632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db29b-30f6-4dcd-97d0-bcbf0222d877 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.116031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db29b-30f6-4dcd-97d0-bcbf0222d877 admin admin] PATH_INFO: `/v3/projects/042f54b443c14c41a280db13076b1fdc/users/c4513148610f4bdaa6793fbeca2feb04/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.117564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 796/1605] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:08.123477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c7c5d-1e2b-4334-b8ff-ab71a7b6df14 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.139631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c7c5d-1e2b-4334-b8ff-ab71a7b6df14 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:08.146884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84d76f-d245-480f-b48c-27b8489f56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:08.147735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d76f-d245-480f-b48c-27b8489f56f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.148025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d76f-d245-480f-b48c-27b8489f56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.148303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d76f-d245-480f-b48c-27b8489f56f7 admin admin] PATH_INFO: `/v3/projects/721722d703af4087bf9bb3854966b482/users/84b6e8d7c0f94bd5a3ad5bf90386b34c/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.148654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402db29b-30f6-4dcd-97d0-bcbf0222d877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=042f54b443c14c41a280db13076b1fdc, user_id=c4513148610f4bdaa6793fbeca2feb04, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:08.152068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402db29b-30f6-4dcd-97d0-bcbf0222d877 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:08.169984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84d76f-d245-480f-b48c-27b8489f56f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=721722d703af4087bf9bb3854966b482, user_id=84b6e8d7c0f94bd5a3ad5bf90386b34c, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:08.173013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84d76f-d245-480f-b48c-27b8489f56f7 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:08.177790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 810/1606] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:07 2022] PUT /identity/v3/projects/042f54b443c14c41a280db13076b1fdc/users/c4513148610f4bdaa6793fbeca2feb04/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:08.179876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815a67c-f44a-46d4-a72b-560f94c5fc4e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.180275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815a67c-f44a-46d4-a72b-560f94c5fc4e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.180686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815a67c-f44a-46d4-a72b-560f94c5fc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.199416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 797/1607] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:08 2022] PUT /identity/v3/projects/721722d703af4087bf9bb3854966b482/users/84b6e8d7c0f94bd5a3ad5bf90386b34c/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:08.205673 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2396-f6fc-4cd1-a316-69a72242bf45 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.205959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2396-f6fc-4cd1-a316-69a72242bf45 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.206243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2396-f6fc-4cd1-a316-69a72242bf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.219117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2815a67c-f44a-46d4-a72b-560f94c5fc4e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:08.220557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2815a67c-f44a-46d4-a72b-560f94c5fc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:08.241574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-d96b2396-f6fc-4cd1-a316-69a72242bf45 None None] MFA Rules not processed for user `e6066712c4604dcfb7407d92230d33d3`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:08.242983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d96b2396-f6fc-4cd1-a316-69a72242bf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:08.252865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:08.321513 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 811/1608] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:08.322813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3fa0-924f-47ce-b830-d3f87238d4a3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.323101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3fa0-924f-47ce-b830-d3f87238d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.324037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3fa0-924f-47ce-b830-d3f87238d4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.348893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 798/1609] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:08.350995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a412d1-8c91-460e-86ea-e1455c2fef68 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.351291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a412d1-8c91-460e-86ea-e1455c2fef68 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.351572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a412d1-8c91-460e-86ea-e1455c2fef68 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.357539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 799/1610] 10.222.0.109 () {54 vars in 967 bytes} [Thu Aug 4 22:20:08 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:20:08.358547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a412d1-8c91-460e-86ea-e1455c2fef68 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.370230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-75de3fa0-924f-47ce-b830-d3f87238d4a3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:08.371589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-75de3fa0-924f-47ce-b830-d3f87238d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:08.377216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a412d1-8c91-460e-86ea-e1455c2fef68 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:08.387382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 812/1611] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:08.388484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62811b-e96d-4443-8b09-408bdf2c53f9 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.388776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62811b-e96d-4443-8b09-408bdf2c53f9 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.389051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62811b-e96d-4443-8b09-408bdf2c53f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.390705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdb235-4287-4820-8261-eaa01f3c5e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:08.391466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb235-4287-4820-8261-eaa01f3c5e7a admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.391850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb235-4287-4820-8261-eaa01f3c5e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.393806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb235-4287-4820-8261-eaa01f3c5e7a admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.394612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdb235-4287-4820-8261-eaa01f3c5e7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:08.397237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdb235-4287-4820-8261-eaa01f3c5e7a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:08.425610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3e62811b-e96d-4443-8b09-408bdf2c53f9 None None] MFA Rules not processed for user `c4513148610f4bdaa6793fbeca2feb04`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:08.426855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3e62811b-e96d-4443-8b09-408bdf2c53f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:08.453855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 800/1612] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/projects => generated 364 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:08.455306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64283896-e895-4456-949c-25fae3f40c5a None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.455666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64283896-e895-4456-949c-25fae3f40c5a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.455946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64283896-e895-4456-949c-25fae3f40c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.477423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-64283896-e895-4456-949c-25fae3f40c5a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:08.478994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-64283896-e895-4456-949c-25fae3f40c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:08.517157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 813/1613] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:08.518919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62811b-e96d-4443-8b09-408bdf2c53f9 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.575164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 801/1614] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:08.577396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b540-1ad1-4344-b328-f03965ddf4a6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.577988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b540-1ad1-4344-b328-f03965ddf4a6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.578448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b540-1ad1-4344-b328-f03965ddf4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.603274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62811b-e96d-4443-8b09-408bdf2c53f9 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:08.641605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-45e4b540-1ad1-4344-b328-f03965ddf4a6 None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:08.642930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-45e4b540-1ad1-4344-b328-f03965ddf4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:08.648412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:08.679638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d5514-88af-4bed-a2fa-f5589a90f424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:08.680282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d5514-88af-4bed-a2fa-f5589a90f424 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.681687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d5514-88af-4bed-a2fa-f5589a90f424 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.682185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d5514-88af-4bed-a2fa-f5589a90f424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.696871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d5514-88af-4bed-a2fa-f5589a90f424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:08.700416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d5514-88af-4bed-a2fa-f5589a90f424 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:08.714312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 814/1615] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:08.719470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d5514-88af-4bed-a2fa-f5589a90f424 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.737690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d5514-88af-4bed-a2fa-f5589a90f424 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:08.747666 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b765b-e1b1-4d26-802e-44596f0b6167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:08.748951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b765b-e1b1-4d26-802e-44596f0b6167 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.750437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b765b-e1b1-4d26-802e-44596f0b6167 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.750698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b765b-e1b1-4d26-802e-44596f0b6167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.761322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 802/1616] 10.222.0.109 () {58 vars in 1064 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:08.762724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455adb3-f802-48c1-b505-4b00ead72339 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.763289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455adb3-f802-48c1-b505-4b00ead72339 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.763289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455adb3-f802-48c1-b505-4b00ead72339 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.767062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 803/1617] 10.222.0.109 () {54 vars in 967 bytes} [Thu Aug 4 22:20:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:20:08.767665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:08.768177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455adb3-f802-48c1-b505-4b00ead72339 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.771592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b765b-e1b1-4d26-802e-44596f0b6167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:08.773153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b765b-e1b1-4d26-802e-44596f0b6167 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:08.795854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455adb3-f802-48c1-b505-4b00ead72339 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:08.798425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 815/1618] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:08.800295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b765b-e1b1-4d26-802e-44596f0b6167 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.808692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9931ff4-c045-4aea-b71b-e1bcd4de0e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:08.809280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931ff4-c045-4aea-b71b-e1bcd4de0e9d admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.809591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931ff4-c045-4aea-b71b-e1bcd4de0e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.809869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931ff4-c045-4aea-b71b-e1bcd4de0e9d admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.812198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9931ff4-c045-4aea-b71b-e1bcd4de0e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:08.816025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9931ff4-c045-4aea-b71b-e1bcd4de0e9d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:08.822840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b765b-e1b1-4d26-802e-44596f0b6167 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:08.835700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f723ff-bd89-405c-89b1-16dd2fc83723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:08.836333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f723ff-bd89-405c-89b1-16dd2fc83723 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.836763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f723ff-bd89-405c-89b1-16dd2fc83723 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.837044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f723ff-bd89-405c-89b1-16dd2fc83723 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.839069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f723ff-bd89-405c-89b1-16dd2fc83723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:08.843362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f723ff-bd89-405c-89b1-16dd2fc83723 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:08.860099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 804/1619] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:08.861077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9931ff4-c045-4aea-b71b-e1bcd4de0e9d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.888978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9931ff4-c045-4aea-b71b-e1bcd4de0e9d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:08.898353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29503d5-7fb6-42e8-bffd-98412af9fd2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:08.899133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29503d5-7fb6-42e8-bffd-98412af9fd2f service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.899411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29503d5-7fb6-42e8-bffd-98412af9fd2f service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.899849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29503d5-7fb6-42e8-bffd-98412af9fd2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.918048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 816/1620] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/projects => generated 374 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:08.918048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f723ff-bd89-405c-89b1-16dd2fc83723 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:08.926781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29503d5-7fb6-42e8-bffd-98412af9fd2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:08.929084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29503d5-7fb6-42e8-bffd-98412af9fd2f service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:08.945480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 805/1621] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:08.946654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccffd7d-8fb0-4e4d-8418-fdb78f028242 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:08.946945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccffd7d-8fb0-4e4d-8418-fdb78f028242 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:08.947223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccffd7d-8fb0-4e4d-8418-fdb78f028242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:08.981995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6ccffd7d-8fb0-4e4d-8418-fdb78f028242 None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:08.983305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6ccffd7d-8fb0-4e4d-8418-fdb78f028242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:09.065335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f723ff-bd89-405c-89b1-16dd2fc83723 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.073795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9308d-3b0f-4a99-abad-0fd2bb2b701b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.074453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9308d-3b0f-4a99-abad-0fd2bb2b701b service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.074920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9308d-3b0f-4a99-abad-0fd2bb2b701b service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.075199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9308d-3b0f-4a99-abad-0fd2bb2b701b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.216857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9308d-3b0f-4a99-abad-0fd2bb2b701b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.216857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9308d-3b0f-4a99-abad-0fd2bb2b701b service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:09.216857 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 817/1622] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:08 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:09.223434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9308d-3b0f-4a99-abad-0fd2bb2b701b service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:09.238712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 806/1623] 10.222.0.109 () {58 vars in 1064 bytes} [Thu Aug 4 22:20:08 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:09.239633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccffd7d-8fb0-4e4d-8418-fdb78f028242 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:09.374993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9308d-3b0f-4a99-abad-0fd2bb2b701b service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.394974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18a396-c954-4978-b260-c6af09b80785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.395690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18a396-c954-4978-b260-c6af09b80785 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.398495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18a396-c954-4978-b260-c6af09b80785 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.398495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18a396-c954-4978-b260-c6af09b80785 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.399051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18a396-c954-4978-b260-c6af09b80785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.401244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18a396-c954-4978-b260-c6af09b80785 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:09.409504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccffd7d-8fb0-4e4d-8418-fdb78f028242 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.425211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711396bd-c33e-4661-88cb-a5ed80cc2024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.429119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711396bd-c33e-4661-88cb-a5ed80cc2024 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.429898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711396bd-c33e-4661-88cb-a5ed80cc2024 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.430887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711396bd-c33e-4661-88cb-a5ed80cc2024 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.434411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711396bd-c33e-4661-88cb-a5ed80cc2024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.440375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711396bd-c33e-4661-88cb-a5ed80cc2024 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:09.451021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 818/1624] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:09 2022] GET /identity/v3/roles => generated 1427 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:09.452788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18a396-c954-4978-b260-c6af09b80785 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:09.466140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:09.472944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18a396-c954-4978-b260-c6af09b80785 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.486512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4ddc6-c3cb-4f18-a19a-93fb61fa6185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.487391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4ddc6-c3cb-4f18-a19a-93fb61fa6185 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.487649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4ddc6-c3cb-4f18-a19a-93fb61fa6185 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.488202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4ddc6-c3cb-4f18-a19a-93fb61fa6185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.497619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 807/1625] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:09 2022] POST /identity/v3/users => generated 294 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:09.498348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:09.499702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711396bd-c33e-4661-88cb-a5ed80cc2024 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:09.525582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711396bd-c33e-4661-88cb-a5ed80cc2024 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.542103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc88d19-e652-4af8-baba-c8edaff404eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.543078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc88d19-e652-4af8-baba-c8edaff404eb admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.543589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc88d19-e652-4af8-baba-c8edaff404eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.543969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc88d19-e652-4af8-baba-c8edaff404eb admin admin] PATH_INFO: `/v3/projects/4ea4325481764e0eb682fea0184e80b3/users/b3e6f334061945948d32865d0d756b66/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.575227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4ddc6-c3cb-4f18-a19a-93fb61fa6185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.577395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4ddc6-c3cb-4f18-a19a-93fb61fa6185 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:09.592015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 819/1626] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:09.592759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4ddc6-c3cb-4f18-a19a-93fb61fa6185 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:09.602715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc88d19-e652-4af8-baba-c8edaff404eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea4325481764e0eb682fea0184e80b3, user_id=b3e6f334061945948d32865d0d756b66, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.605470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc88d19-e652-4af8-baba-c8edaff404eb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:09.608227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4ddc6-c3cb-4f18-a19a-93fb61fa6185 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.615332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59952846-3ff6-4485-b568-18ee814514e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.616439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59952846-3ff6-4485-b568-18ee814514e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.616903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59952846-3ff6-4485-b568-18ee814514e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.617607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59952846-3ff6-4485-b568-18ee814514e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.618856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59952846-3ff6-4485-b568-18ee814514e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.621234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59952846-3ff6-4485-b568-18ee814514e4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:09.625758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 808/1627] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:09 2022] PUT /identity/v3/projects/4ea4325481764e0eb682fea0184e80b3/users/b3e6f334061945948d32865d0d756b66/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:09.629692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc88d19-e652-4af8-baba-c8edaff404eb admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:09.644570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 820/1628] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:09 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:09.646562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59952846-3ff6-4485-b568-18ee814514e4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:09.661291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59952846-3ff6-4485-b568-18ee814514e4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.745373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc88d19-e652-4af8-baba-c8edaff404eb admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.751233 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26e120-821a-4799-b2e4-d8ab2cf5b30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.751646 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7fee1-fd17-4555-950c-18a35b5453e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.752180 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26e120-821a-4799-b2e4-d8ab2cf5b30a admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.752470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26e120-821a-4799-b2e4-d8ab2cf5b30a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.752751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26e120-821a-4799-b2e4-d8ab2cf5b30a admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.752934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7fee1-fd17-4555-950c-18a35b5453e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.753105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7fee1-fd17-4555-950c-18a35b5453e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.753383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7fee1-fd17-4555-950c-18a35b5453e3 admin admin] PATH_INFO: `/v3/projects/c6ac68040f144d7abcd417d0231370d0/users/8df0bb5f50284c35814513d8cefa002f/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.754172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26e120-821a-4799-b2e4-d8ab2cf5b30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.757024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26e120-821a-4799-b2e4-d8ab2cf5b30a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:09.771923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 809/1629] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:09 2022] GET /identity/v3/roles => generated 1427 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:20:09.774335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337a4c-9bcb-4544-b7b9-4794a3d3d10f None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.774677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337a4c-9bcb-4544-b7b9-4794a3d3d10f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.775001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337a4c-9bcb-4544-b7b9-4794a3d3d10f None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.776609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7fee1-fd17-4555-950c-18a35b5453e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ac68040f144d7abcd417d0231370d0, user_id=8df0bb5f50284c35814513d8cefa002f, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.777610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 810/1630] 10.222.0.109 () {54 vars in 917 bytes} [Thu Aug 4 22:20:09 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:20:09.779670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7fee1-fd17-4555-950c-18a35b5453e3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:09.782921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337a4c-9bcb-4544-b7b9-4794a3d3d10f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:09.806803 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 821/1631] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:09 2022] PUT /identity/v3/projects/c6ac68040f144d7abcd417d0231370d0/users/8df0bb5f50284c35814513d8cefa002f/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:20:09.810322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f3735-ad04-4b56-bb2f-880e41ac1c71 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.810322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f3735-ad04-4b56-bb2f-880e41ac1c71 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.810322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f3735-ad04-4b56-bb2f-880e41ac1c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.815307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337a4c-9bcb-4544-b7b9-4794a3d3d10f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:09.858381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-741f3735-ad04-4b56-bb2f-880e41ac1c71 None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:09.860642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-741f3735-ad04-4b56-bb2f-880e41ac1c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:09.972271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5bbcd-b9ad-4dee-8b57-9c1dc2be4cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:09.972973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5bbcd-b9ad-4dee-8b57-9c1dc2be4cf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:09.973258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5bbcd-b9ad-4dee-8b57-9c1dc2be4cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:09.974634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5bbcd-b9ad-4dee-8b57-9c1dc2be4cf6 admin admin] PATH_INFO: `/v3/projects/4ea4325481764e0eb682fea0184e80b3/users/b3e6f334061945948d32865d0d756b66/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:09.980310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5bbcd-b9ad-4dee-8b57-9c1dc2be4cf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea4325481764e0eb682fea0184e80b3, user_id=b3e6f334061945948d32865d0d756b66, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:09.983214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5bbcd-b9ad-4dee-8b57-9c1dc2be4cf6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:10.011981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 811/1632] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:09 2022] PUT /identity/v3/projects/4ea4325481764e0eb682fea0184e80b3/users/b3e6f334061945948d32865d0d756b66/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:10.013045 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5bbcd-b9ad-4dee-8b57-9c1dc2be4cf6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:10.031007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 822/1633] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:20:09 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:10.033082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210f54b-df9f-4f23-9bed-4b24024d4329 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.033378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210f54b-df9f-4f23-9bed-4b24024d4329 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.033687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210f54b-df9f-4f23-9bed-4b24024d4329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:10.060720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:10.074164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4210f54b-df9f-4f23-9bed-4b24024d4329 None None] MFA Rules not processed for user `b3e6f334061945948d32865d0d756b66`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:10.075588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4210f54b-df9f-4f23-9bed-4b24024d4329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:10.173623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 823/1634] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:10.174639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210f54b-df9f-4f23-9bed-4b24024d4329 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:10.190486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:10.199371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5bbcd-b9ad-4dee-8b57-9c1dc2be4cf6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:10.213227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f87860-ba75-4bdd-b1fb-e760888c15b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:10.214087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f87860-ba75-4bdd-b1fb-e760888c15b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.214444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f87860-ba75-4bdd-b1fb-e760888c15b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.214924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f87860-ba75-4bdd-b1fb-e760888c15b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:10.217061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f87860-ba75-4bdd-b1fb-e760888c15b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:10.219350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f87860-ba75-4bdd-b1fb-e760888c15b3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:10.246433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 812/1635] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:10 2022] GET /identity/v3/roles => generated 1427 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:10.247957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a7d7b-01b3-4821-9f1b-404af112ebf6 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.249985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a7d7b-01b3-4821-9f1b-404af112ebf6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.252580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a7d7b-01b3-4821-9f1b-404af112ebf6 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:10.257243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 813/1636] 10.222.0.109 () {54 vars in 917 bytes} [Thu Aug 4 22:20:10 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:20:10.258361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a7d7b-01b3-4821-9f1b-404af112ebf6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:10.293946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210f54b-df9f-4f23-9bed-4b24024d4329 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:10.303312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86bc55-3255-4b4f-8448-68f6e706d7a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:10.304622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86bc55-3255-4b4f-8448-68f6e706d7a1 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.305090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86bc55-3255-4b4f-8448-68f6e706d7a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.306181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86bc55-3255-4b4f-8448-68f6e706d7a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:10.357853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a7d7b-01b3-4821-9f1b-404af112ebf6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:10.367456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de1853-4cee-4ff6-bd7a-3cc1f388d9cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:10.368687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de1853-4cee-4ff6-bd7a-3cc1f388d9cd service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.369048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de1853-4cee-4ff6-bd7a-3cc1f388d9cd service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.369927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de1853-4cee-4ff6-bd7a-3cc1f388d9cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:10.389889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de1853-4cee-4ff6-bd7a-3cc1f388d9cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:10.391995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de1853-4cee-4ff6-bd7a-3cc1f388d9cd service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:10.395637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86bc55-3255-4b4f-8448-68f6e706d7a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:10.397984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86bc55-3255-4b4f-8448-68f6e706d7a1 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:10.413125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 814/1637] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:10.414275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de1853-4cee-4ff6-bd7a-3cc1f388d9cd service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:10.426831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 824/1638] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:10.429133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de1853-4cee-4ff6-bd7a-3cc1f388d9cd service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:10.434436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d1860-69a9-4b1b-8519-83e90bc5529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:10.434936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae40a-7776-4e21-b55d-8562c040d3cf None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.435620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1860-69a9-4b1b-8519-83e90bc5529f admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.436007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1860-69a9-4b1b-8519-83e90bc5529f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.436271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae40a-7776-4e21-b55d-8562c040d3cf None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.437069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1860-69a9-4b1b-8519-83e90bc5529f admin admin] PATH_INFO: `/v3/projects/c6ac68040f144d7abcd417d0231370d0/users/8df0bb5f50284c35814513d8cefa002f/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:10.437675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae40a-7776-4e21-b55d-8562c040d3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:10.439254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d1860-69a9-4b1b-8519-83e90bc5529f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ac68040f144d7abcd417d0231370d0, user_id=8df0bb5f50284c35814513d8cefa002f, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:10.442670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d1860-69a9-4b1b-8519-83e90bc5529f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:10.469116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 815/1639] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:10 2022] PUT /identity/v3/projects/c6ac68040f144d7abcd417d0231370d0/users/8df0bb5f50284c35814513d8cefa002f/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:10.472226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bac44-f6ec-42e4-9025-e9f96f0cfa45 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.472597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bac44-f6ec-42e4-9025-e9f96f0cfa45 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.473056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bac44-f6ec-42e4-9025-e9f96f0cfa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:10.492376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-47eae40a-7776-4e21-b55d-8562c040d3cf None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:10.495074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-47eae40a-7776-4e21-b55d-8562c040d3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:10.549098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f10bac44-f6ec-42e4-9025-e9f96f0cfa45 None None] MFA Rules not processed for user `8df0bb5f50284c35814513d8cefa002f`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:10.552490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f10bac44-f6ec-42e4-9025-e9f96f0cfa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:10.634240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:10.760836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 825/1640] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:20:10 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:10.763567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eae40a-7776-4e21-b55d-8562c040d3cf None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:10.816438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 816/1641] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:10.920692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eae40a-7776-4e21-b55d-8562c040d3cf None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:10.933948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3b162-ff64-4963-8b6f-32b5031ad668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:10.935793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3b162-ff64-4963-8b6f-32b5031ad668 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:10.936081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3b162-ff64-4963-8b6f-32b5031ad668 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:10.939550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3b162-ff64-4963-8b6f-32b5031ad668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:11.078690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3b162-ff64-4963-8b6f-32b5031ad668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:11.080332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3b162-ff64-4963-8b6f-32b5031ad668 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:11.125877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 826/1642] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:11.399900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bac44-f6ec-42e4-9025-e9f96f0cfa45 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:11.412859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bac44-f6ec-42e4-9025-e9f96f0cfa45 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:11.424946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304aabf8-4d0e-4775-9a1b-45322d6a3aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:11.425665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304aabf8-4d0e-4775-9a1b-45322d6a3aed service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:11.427866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304aabf8-4d0e-4775-9a1b-45322d6a3aed service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:11.428093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304aabf8-4d0e-4775-9a1b-45322d6a3aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:11.456178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304aabf8-4d0e-4775-9a1b-45322d6a3aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:11.460275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304aabf8-4d0e-4775-9a1b-45322d6a3aed service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:11.502680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 817/1643] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:11 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:11.573930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3b162-ff64-4963-8b6f-32b5031ad668 service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:11.583606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3b162-ff64-4963-8b6f-32b5031ad668 service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:11.594629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8bb4e-b6a7-4b79-8079-a6da3b868a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:11.596207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bb4e-b6a7-4b79-8079-a6da3b868a64 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:11.596936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bb4e-b6a7-4b79-8079-a6da3b868a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:11.600950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bb4e-b6a7-4b79-8079-a6da3b868a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:11.623200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8bb4e-b6a7-4b79-8079-a6da3b868a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:11.628521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8bb4e-b6a7-4b79-8079-a6da3b868a64 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:11.669158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 827/1644] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:12.324704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304aabf8-4d0e-4775-9a1b-45322d6a3aed service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:12.351542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304aabf8-4d0e-4775-9a1b-45322d6a3aed service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:12.367880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432648e-00c7-4154-8a96-aa5b1884b878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:12.368521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432648e-00c7-4154-8a96-aa5b1884b878 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:12.368870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432648e-00c7-4154-8a96-aa5b1884b878 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:12.369164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432648e-00c7-4154-8a96-aa5b1884b878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:12.467398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:12.470963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432648e-00c7-4154-8a96-aa5b1884b878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:12.479744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432648e-00c7-4154-8a96-aa5b1884b878 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:12.494926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 818/1645] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:15.815718 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8bb4e-b6a7-4b79-8079-a6da3b868a64 service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:15.975209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8bb4e-b6a7-4b79-8079-a6da3b868a64 service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:15.990810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:15.991583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:15.991938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:15.992292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] PATH_INFO: `/v3/users/337832e987334f5d8def83bca7331497` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:15.994474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] RBAC: Authorizing `identity:delete_user(user_id=337832e987334f5d8def83bca7331497)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:15.996705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:16.019848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:16.122106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '337832e987334f5d8def83bca7331497'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:16.135044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '337832e987334f5d8def83bca7331497'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:16.158501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '337832e987334f5d8def83bca7331497'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:16.160285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '337832e987334f5d8def83bca7331497'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:16.208988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '337832e987334f5d8def83bca7331497'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:16.222408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 828/1646] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:15 2022] DELETE /identity/v3/users/337832e987334f5d8def83bca7331497 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:16.227513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a629-9de6-47b7-adb6-9b61f96406a6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:16.228481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a629-9de6-47b7-adb6-9b61f96406a6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:16.228982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a629-9de6-47b7-adb6-9b61f96406a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:16.267379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-1d62a629-9de6-47b7-adb6-9b61f96406a6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:16.268662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1d62a629-9de6-47b7-adb6-9b61f96406a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:16.430937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 819/1647] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:16.479093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:16.480848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:16.614460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d968285-5d78-42df-9200-59de4f6b57db None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:16.614825 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d968285-5d78-42df-9200-59de4f6b57db None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:16.615106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d968285-5d78-42df-9200-59de4f6b57db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:16.615995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac317e0-2826-4dc0-aeeb-a77163d0386f admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:16.626507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:16.628619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:16.631631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:16.632052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:16.651131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:16.659545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:16.694785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 829/1648] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:16.697003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0d968285-5d78-42df-9200-59de4f6b57db None None] MFA Rules not processed for user `8c0aa2b9a5ac46f9bede8d007dcd3504`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:16.698494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0d968285-5d78-42df-9200-59de4f6b57db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:16.845289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 820/1649] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:16.859487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:16.861510 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:17.088398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60605fd-be56-4464-8c28-fde040dfa5f4 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:17.108143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efb424-4516-4d43-bc90-a4d6044e8077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:17.108949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efb424-4516-4d43-bc90-a4d6044e8077 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:17.109368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efb424-4516-4d43-bc90-a4d6044e8077 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:17.109763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efb424-4516-4d43-bc90-a4d6044e8077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:17.150527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efb424-4516-4d43-bc90-a4d6044e8077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:17.151047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efb424-4516-4d43-bc90-a4d6044e8077 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:17.175043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 830/1650] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:18.318318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d968285-5d78-42df-9200-59de4f6b57db None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:18.320008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0d968285-5d78-42df-9200-59de4f6b57db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:18.357332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d968285-5d78-42df-9200-59de4f6b57db None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:18.372913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:18.373646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:18.373931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:18.374203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] PATH_INFO: `/v3/projects/92c96f8280d94bf4b15c7fdbad9b17ac` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:18.378382 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c96f8280d94bf4b15c7fdbad9b17ac)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:18.380905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:18.512259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c96f8280d94bf4b15c7fdbad9b17ac', 'request_id': 'req-e50010ce-5beb-4560-baa9-2647c34c0fb3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:18.529860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c96f8280d94bf4b15c7fdbad9b17ac', 'request_id': 'req-e50010ce-5beb-4560-baa9-2647c34c0fb3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:18.555931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c96f8280d94bf4b15c7fdbad9b17ac', 'request_id': 'req-e50010ce-5beb-4560-baa9-2647c34c0fb3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:18.577815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] The token cache is being invalidate because project 92c96f8280d94bf4b15c7fdbad9b17ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:18.578001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:18.585157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e50010ce-5beb-4560-baa9-2647c34c0fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:18.590441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 821/1651] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:18 2022] DELETE /identity/v3/projects/92c96f8280d94bf4b15c7fdbad9b17ac => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:18.594791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efb424-4516-4d43-bc90-a4d6044e8077 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:18.638891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efb424-4516-4d43-bc90-a4d6044e8077 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:18.690844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:18.861117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:18.863418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:18.864625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:18.865066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] PATH_INFO: `/v3/users/b3e6f334061945948d32865d0d756b66` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:18.867815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e6f334061945948d32865d0d756b66)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:18.875945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:19.034064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e6f334061945948d32865d0d756b66'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:19.059608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e6f334061945948d32865d0d756b66'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:19.077166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e6f334061945948d32865d0d756b66'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:19.080305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e6f334061945948d32865d0d756b66'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:19.176321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76460f-5374-4b12-bdff-10b4e6b2539d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e6f334061945948d32865d0d756b66'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:19.216685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 831/1652] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:18 2022] DELETE /identity/v3/users/b3e6f334061945948d32865d0d756b66 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:19.577434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2175-6fb5-4a74-8fe8-cc8d771809a9 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:19.577940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2175-6fb5-4a74-8fe8-cc8d771809a9 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:19.578060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2175-6fb5-4a74-8fe8-cc8d771809a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:19.603103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:19.641716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-030c2175-6fb5-4a74-8fe8-cc8d771809a9 None None] MFA Rules not processed for user `84b6e8d7c0f94bd5a3ad5bf90386b34c`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:19.642996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-030c2175-6fb5-4a74-8fe8-cc8d771809a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:19.824767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 822/1653] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:19.826220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c73a5-0164-470a-870d-ae5427a4abbe None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:19.826718 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c73a5-0164-470a-870d-ae5427a4abbe None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:19.826826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c73a5-0164-470a-870d-ae5427a4abbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:19.869015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a64c73a5-0164-470a-870d-ae5427a4abbe None None] MFA Rules not processed for user `84b6e8d7c0f94bd5a3ad5bf90386b34c`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:19.878129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a64c73a5-0164-470a-870d-ae5427a4abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:19.918687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 832/1654] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:19.931110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2175-6fb5-4a74-8fe8-cc8d771809a9 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:19.932565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-030c2175-6fb5-4a74-8fe8-cc8d771809a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:20.204543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2175-6fb5-4a74-8fe8-cc8d771809a9 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:20.228869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898aaea-8cb8-4794-a4c5-4583ff77d04a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:20.230731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898aaea-8cb8-4794-a4c5-4583ff77d04a service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:20.231564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898aaea-8cb8-4794-a4c5-4583ff77d04a service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:20.231564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898aaea-8cb8-4794-a4c5-4583ff77d04a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:20.265391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898aaea-8cb8-4794-a4c5-4583ff77d04a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:20.271913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898aaea-8cb8-4794-a4c5-4583ff77d04a service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:20.316065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 823/1655] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:21.264199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c73a5-0164-470a-870d-ae5427a4abbe None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:21.266465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a64c73a5-0164-470a-870d-ae5427a4abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:21.429783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c73a5-0164-470a-870d-ae5427a4abbe None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:21.451081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:21.455875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:21.462898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:21.464603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] PATH_INFO: `/v3/projects/4ea4325481764e0eb682fea0184e80b3` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:21.469610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea4325481764e0eb682fea0184e80b3)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:21.470935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: WARNING py.warnings [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:20:21.470935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: warnings.warn(msg) Aug 04 22:20:21.470935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: Aug 04 22:20:21.474958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:21.496366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6fc39-9a00-4bc6-b280-efd3ab4b43b6 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:21.496862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6fc39-9a00-4bc6-b280-efd3ab4b43b6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:21.496954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6fc39-9a00-4bc6-b280-efd3ab4b43b6 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:21.498337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 824/1656] 10.222.0.109 () {54 vars in 917 bytes} [Thu Aug 4 22:20:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:20:21.524146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da659a-5234-4860-ac14-66773441b46a None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:21.524542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da659a-5234-4860-ac14-66773441b46a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:21.524822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da659a-5234-4860-ac14-66773441b46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:21.622660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:21.643752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-51da659a-5234-4860-ac14-66773441b46a None None] MFA Rules not processed for user `5455c117e349480ba8b8c020b767bec5`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:21.644990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-51da659a-5234-4860-ac14-66773441b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:21.681307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea4325481764e0eb682fea0184e80b3', 'request_id': 'req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:21.702628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea4325481764e0eb682fea0184e80b3', 'request_id': 'req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:21.742147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea4325481764e0eb682fea0184e80b3', 'request_id': 'req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:21.782413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] The token cache is being invalidate because project 4ea4325481764e0eb682fea0184e80b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:21.782952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:21.783784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c69a09c-b1e4-47ab-a148-f7e21dd51e9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:21.794351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 833/1657] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:21 2022] DELETE /identity/v3/projects/4ea4325481764e0eb682fea0184e80b3 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:21.796107 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d630a1c-cafd-4065-b176-df77b132c80f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:21.796320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d630a1c-cafd-4065-b176-df77b132c80f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:21.797537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d630a1c-cafd-4065-b176-df77b132c80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:21.820969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 825/1658] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:20:21 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:21.865796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4d630a1c-cafd-4065-b176-df77b132c80f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:21.876470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4d630a1c-cafd-4065-b176-df77b132c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:21.993048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 834/1659] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:21.993362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933cf67-d3a1-4c6d-866f-aa3ac7e679b2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:21.994503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933cf67-d3a1-4c6d-866f-aa3ac7e679b2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:21.994915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933cf67-d3a1-4c6d-866f-aa3ac7e679b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.033917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9933cf67-d3a1-4c6d-866f-aa3ac7e679b2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:22.036515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9933cf67-d3a1-4c6d-866f-aa3ac7e679b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:22.060805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 826/1660] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:22.063842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d630a1c-cafd-4065-b176-df77b132c80f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.090536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d630a1c-cafd-4065-b176-df77b132c80f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:22.099945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:22.101561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.101561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.101742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.102976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:22.103286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:22.105331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:22.124296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933cf67-d3a1-4c6d-866f-aa3ac7e679b2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.127353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9933cf67-d3a1-4c6d-866f-aa3ac7e679b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:22.152065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 835/1661] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:20:22.159662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.167867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933cf67-d3a1-4c6d-866f-aa3ac7e679b2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:22.182632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:22.185887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.189806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.191144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.200352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad127f3a-c544-4152-8c3a-12070b60be3d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:22.226803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:22.227707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccddd04-dbc7-41eb-87ab-9281b818de98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:22.227707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccddd04-dbc7-41eb-87ab-9281b818de98 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.227707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccddd04-dbc7-41eb-87ab-9281b818de98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.228753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccddd04-dbc7-41eb-87ab-9281b818de98 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.229243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:22.229564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccddd04-dbc7-41eb-87ab-9281b818de98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:22.234434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccddd04-dbc7-41eb-87ab-9281b818de98 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:22.249110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:22.253876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 827/1662] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:22 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:22.257582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.260634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:22.288674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69443413-aa69-42a1-a01f-61977e002159 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:22.295443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:22.296106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.296385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.296655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] PATH_INFO: `/v3/users/e6066712c4604dcfb7407d92230d33d3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.299236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6066712c4604dcfb7407d92230d33d3)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:22.300719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: WARNING py.warnings [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 22:20:22.300719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: warnings.warn(msg) Aug 04 22:20:22.300719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: Aug 04 22:20:22.302743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:22.343171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 836/1663] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:22 2022] POST /identity/v3/projects => generated 386 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:22.344066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccddd04-dbc7-41eb-87ab-9281b818de98 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.449969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6066712c4604dcfb7407d92230d33d3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:22.465344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6066712c4604dcfb7407d92230d33d3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:22.466968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6066712c4604dcfb7407d92230d33d3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:22.517274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccddd04-dbc7-41eb-87ab-9281b818de98 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:22.520420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6066712c4604dcfb7407d92230d33d3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:22.521876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1a897b79-6a16-40ea-a76e-ea6c4496e1cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6066712c4604dcfb7407d92230d33d3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:22.530156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:22.530799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.531082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.531357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.532040 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:22.534173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:22.569245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 828/1664] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:22 2022] DELETE /identity/v3/users/e6066712c4604dcfb7407d92230d33d3 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:22.573523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7903f89-a04b-46a1-b07a-72761a77ba83 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.574009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7903f89-a04b-46a1-b07a-72761a77ba83 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.574833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7903f89-a04b-46a1-b07a-72761a77ba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.603430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 837/1665] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:22 2022] POST /identity/v3/users => generated 294 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:22.607151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.610024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:22.634506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-d7903f89-a04b-46a1-b07a-72761a77ba83 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:22.637537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d7903f89-a04b-46a1-b07a-72761a77ba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:22.638859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68c81-b144-48aa-bbb1-67c902bbd49b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:22.674732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:22.798481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9a0d8-dc2a-4b10-87ff-4bf9526bd8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:22.799255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a0d8-dc2a-4b10-87ff-4bf9526bd8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.799636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a0d8-dc2a-4b10-87ff-4bf9526bd8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.800009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9a0d8-dc2a-4b10-87ff-4bf9526bd8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.802979 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9a0d8-dc2a-4b10-87ff-4bf9526bd8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:22.804829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9a0d8-dc2a-4b10-87ff-4bf9526bd8f4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:22.825831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 838/1666] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:22 2022] GET /identity/v3/roles => generated 1427 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:22.828643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 829/1667] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:22.830069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9a0d8-dc2a-4b10-87ff-4bf9526bd8f4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.848645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9a0d8-dc2a-4b10-87ff-4bf9526bd8f4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:22.867508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:22.868382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.868864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.869332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.870290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:22.872127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:22.892692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7903f89-a04b-46a1-b07a-72761a77ba83 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.902324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 839/1668] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:22 2022] GET /identity/v3/roles => generated 1427 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:20:22.903652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:22.908436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7903f89-a04b-46a1-b07a-72761a77ba83 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:22.908813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:22.915975 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6683d-15b8-497d-939e-b7e86a525af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:22.916867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6683d-15b8-497d-939e-b7e86a525af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:22.917254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6683d-15b8-497d-939e-b7e86a525af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:22.917659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6683d-15b8-497d-939e-b7e86a525af6 admin admin] PATH_INFO: `/v3/projects/d885cc270f764ae289330459966bddbc/users/4a34d6938614463883e4b5fdfc976b49/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:22.962906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6683d-15b8-497d-939e-b7e86a525af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d885cc270f764ae289330459966bddbc, user_id=4a34d6938614463883e4b5fdfc976b49, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:22.970943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6683d-15b8-497d-939e-b7e86a525af6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:23.003422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 830/1669] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:22 2022] PUT /identity/v3/projects/d885cc270f764ae289330459966bddbc/users/4a34d6938614463883e4b5fdfc976b49/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:23.013646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21c7c2-0444-49b4-ae83-b7697cbf15e0 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:23.014034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21c7c2-0444-49b4-ae83-b7697cbf15e0 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:23.014714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21c7c2-0444-49b4-ae83-b7697cbf15e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:23.051905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81fa38-9b12-45ec-9c12-7e4ea6691184 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:23.091815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6d21c7c2-0444-49b4-ae83-b7697cbf15e0 None None] MFA Rules not processed for user `4a34d6938614463883e4b5fdfc976b49`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:23.093085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6d21c7c2-0444-49b4-ae83-b7697cbf15e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:23.169965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120b5b4-594f-41e7-943c-61d6eeb453f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:23.170610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b5b4-594f-41e7-943c-61d6eeb453f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:23.170891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b5b4-594f-41e7-943c-61d6eeb453f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:23.171169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b5b4-594f-41e7-943c-61d6eeb453f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:23.224216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:23.298759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 831/1670] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:23.303245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120b5b4-594f-41e7-943c-61d6eeb453f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:23.304267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120b5b4-594f-41e7-943c-61d6eeb453f7 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:23.306299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21c7c2-0444-49b4-ae83-b7697cbf15e0 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:23.337041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 840/1671] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:23.355178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21c7c2-0444-49b4-ae83-b7697cbf15e0 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:23.360806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e3186-d7f9-42f4-9d55-b2aa066f38b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:23.361739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3186-d7f9-42f4-9d55-b2aa066f38b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:23.362163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3186-d7f9-42f4-9d55-b2aa066f38b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:23.362581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3186-d7f9-42f4-9d55-b2aa066f38b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:23.363422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e3186-d7f9-42f4-9d55-b2aa066f38b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:23.369843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e3186-d7f9-42f4-9d55-b2aa066f38b2 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:23.422250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 832/1672] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:23 2022] POST /identity/v3/projects => generated 388 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:23.425949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120b5b4-594f-41e7-943c-61d6eeb453f7 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:23.441329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d21e6e-e7cd-4928-9b49-d72da74d7679 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:23.441706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d21e6e-e7cd-4928-9b49-d72da74d7679 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:23.442577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:23.446647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d21e6e-e7cd-4928-9b49-d72da74d7679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:23.451694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120b5b4-594f-41e7-943c-61d6eeb453f7 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:23.483661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-35d21e6e-e7cd-4928-9b49-d72da74d7679 None None] MFA Rules not processed for user `8c0aa2b9a5ac46f9bede8d007dcd3504`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:23.486178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-35d21e6e-e7cd-4928-9b49-d72da74d7679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:23.583205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40d74-4c1a-49ac-8106-cf5f9662626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:23.584003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40d74-4c1a-49ac-8106-cf5f9662626f admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:23.584504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40d74-4c1a-49ac-8106-cf5f9662626f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:23.584848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40d74-4c1a-49ac-8106-cf5f9662626f admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:23.585776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f40d74-4c1a-49ac-8106-cf5f9662626f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:23.591373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f40d74-4c1a-49ac-8106-cf5f9662626f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:23.621579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 833/1673] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:23.656022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 841/1674] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:23 2022] POST /identity/v3/users => generated 301 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:23.660098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d21e6e-e7cd-4928-9b49-d72da74d7679 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:23.674122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d21e6e-e7cd-4928-9b49-d72da74d7679 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:23.680717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1db53-388e-4474-ae80-0a3f5782e4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:23.682034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1db53-388e-4474-ae80-0a3f5782e4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:23.682526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1db53-388e-4474-ae80-0a3f5782e4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:23.683015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1db53-388e-4474-ae80-0a3f5782e4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:23.683953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1db53-388e-4474-ae80-0a3f5782e4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:23.686418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1db53-388e-4474-ae80-0a3f5782e4f7 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:23.701931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 834/1675] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:23 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:23.706442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40d74-4c1a-49ac-8106-cf5f9662626f admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:23.731026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40d74-4c1a-49ac-8106-cf5f9662626f admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:23.736228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ff2e9-2db8-4675-ab39-ce166786d745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:23.737531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff2e9-2db8-4675-ab39-ce166786d745 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:23.739781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff2e9-2db8-4675-ab39-ce166786d745 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:23.740011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff2e9-2db8-4675-ab39-ce166786d745 admin admin] PATH_INFO: `/v3/projects/469f1d1ea08b4d2fab96d575a070d977/users/394287cf90934789803be4f51aa37d78/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:23.765919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ff2e9-2db8-4675-ab39-ce166786d745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=469f1d1ea08b4d2fab96d575a070d977, user_id=394287cf90934789803be4f51aa37d78, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:23.770655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ff2e9-2db8-4675-ab39-ce166786d745 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:23.789318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 842/1676] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:23 2022] PUT /identity/v3/projects/469f1d1ea08b4d2fab96d575a070d977/users/394287cf90934789803be4f51aa37d78/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:23.793065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1db53-388e-4474-ae80-0a3f5782e4f7 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:23.809872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1db53-388e-4474-ae80-0a3f5782e4f7 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:24.021860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d77b4-8a87-4b84-8ae2-20156c26d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:24.022526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d77b4-8a87-4b84-8ae2-20156c26d8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.022812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d77b4-8a87-4b84-8ae2-20156c26d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.023092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d77b4-8a87-4b84-8ae2-20156c26d8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.023814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d77b4-8a87-4b84-8ae2-20156c26d8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:24.025492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d77b4-8a87-4b84-8ae2-20156c26d8a8 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:24.045000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71139-ff96-46c7-91c9-4994d6053f1e None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.045590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71139-ff96-46c7-91c9-4994d6053f1e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.046104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71139-ff96-46c7-91c9-4994d6053f1e None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.048047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 843/1677] 10.222.0.109 () {54 vars in 966 bytes} [Thu Aug 4 22:20:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:20:24.055264 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d41fa-905d-47e6-8f7e-938f1e2a25b7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.056056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d41fa-905d-47e6-8f7e-938f1e2a25b7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.056507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d41fa-905d-47e6-8f7e-938f1e2a25b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.057002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 835/1678] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:23 2022] GET /identity/v3/roles => generated 1427 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:24.063660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d77b4-8a87-4b84-8ae2-20156c26d8a8 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:24.085092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-fa9d41fa-905d-47e6-8f7e-938f1e2a25b7 None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:24.086882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fa9d41fa-905d-47e6-8f7e-938f1e2a25b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:24.087156 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d77b4-8a87-4b84-8ae2-20156c26d8a8 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:24.097362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a464e9-8180-44f4-8b20-4fa527446696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:24.098588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a464e9-8180-44f4-8b20-4fa527446696 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.113247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a464e9-8180-44f4-8b20-4fa527446696 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.115439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a464e9-8180-44f4-8b20-4fa527446696 admin admin] PATH_INFO: `/v3/projects/469f1d1ea08b4d2fab96d575a070d977/users/394287cf90934789803be4f51aa37d78/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.120127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a464e9-8180-44f4-8b20-4fa527446696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=469f1d1ea08b4d2fab96d575a070d977, user_id=394287cf90934789803be4f51aa37d78, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:24.142812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a464e9-8180-44f4-8b20-4fa527446696 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:24.196939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 836/1679] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:24 2022] PUT /identity/v3/projects/469f1d1ea08b4d2fab96d575a070d977/users/394287cf90934789803be4f51aa37d78/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:24.208089 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02559dc5-0472-410f-b30e-719a1181b48c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.208469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02559dc5-0472-410f-b30e-719a1181b48c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.208821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02559dc5-0472-410f-b30e-719a1181b48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.239064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 844/1680] 10.222.0.109 () {58 vars in 1063 bytes} [Thu Aug 4 22:20:24 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:24.248987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d41fa-905d-47e6-8f7e-938f1e2a25b7 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:24.264331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d41fa-905d-47e6-8f7e-938f1e2a25b7 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:24.289052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-02559dc5-0472-410f-b30e-719a1181b48c None None] MFA Rules not processed for user `394287cf90934789803be4f51aa37d78`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:24.289052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-02559dc5-0472-410f-b30e-719a1181b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:24.351932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:24.412904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:24.413575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.413899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.414174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.424328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:24.435846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:24.439637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:24.442438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:24.455181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 845/1681] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:24 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:24.530051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 837/1682] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:24 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:24.531611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:24.533564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:24.614922 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e072410-ba07-42a5-9ccc-1e2ce8754531 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.615328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e072410-ba07-42a5-9ccc-1e2ce8754531 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.615712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e072410-ba07-42a5-9ccc-1e2ce8754531 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.617489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 838/1683] 10.222.0.109 () {54 vars in 917 bytes} [Thu Aug 4 22:20:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 22:20:24.623710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f37c90-bdbf-48d4-aa6b-434d3de022b8 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.624350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f37c90-bdbf-48d4-aa6b-434d3de022b8 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.624350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f37c90-bdbf-48d4-aa6b-434d3de022b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.646908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f3f37c90-bdbf-48d4-aa6b-434d3de022b8 None None] MFA Rules not processed for user `5455c117e349480ba8b8c020b767bec5`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:24.648394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f3f37c90-bdbf-48d4-aa6b-434d3de022b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:24.664697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e778e-0d3e-4d95-ab17-5b854ffdaecc service placement] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:24.669330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40fc07-b9d0-4d8d-93db-7fa608b744f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:24.670050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40fc07-b9d0-4d8d-93db-7fa608b744f4 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.670331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40fc07-b9d0-4d8d-93db-7fa608b744f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.670608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40fc07-b9d0-4d8d-93db-7fa608b744f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.689745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40fc07-b9d0-4d8d-93db-7fa608b744f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:24.691133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40fc07-b9d0-4d8d-93db-7fa608b744f4 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:24.710169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 846/1684] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:24 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:20:24.773003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40fc07-b9d0-4d8d-93db-7fa608b744f4 service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:24.820735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 839/1685] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:20:24 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:24.833928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f37c90-bdbf-48d4-aa6b-434d3de022b8 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:24.859315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f37c90-bdbf-48d4-aa6b-434d3de022b8 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:24.903795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:24.904465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:24.904739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:24.909780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:24.961058 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40fc07-b9d0-4d8d-93db-7fa608b744f4 service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:24.976199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:24.978695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:24.993592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 840/1686] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:24 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:20:24.998023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:25.001638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:25.001993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:25.002395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] PATH_INFO: `/v3/users/4a34d6938614463883e4b5fdfc976b49` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:25.005993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a34d6938614463883e4b5fdfc976b49)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:25.007658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:25.092090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:25.093287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:25.153868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1cfcf-df9c-4256-85b8-7ac5450e66a6 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:25.171736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:25.175462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:25.175750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:25.176080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] PATH_INFO: `/v3/projects/c940df02ebff4e94bf0874df846db424` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:25.190711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c940df02ebff4e94bf0874df846db424)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:25.200235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:25.211742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a34d6938614463883e4b5fdfc976b49'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.235761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a34d6938614463883e4b5fdfc976b49'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.260034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a34d6938614463883e4b5fdfc976b49'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.260748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a34d6938614463883e4b5fdfc976b49'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.296308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c627aa5b-aeab-4622-9f61-ee7cb091c41f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a34d6938614463883e4b5fdfc976b49'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.305031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 847/1687] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:24 2022] DELETE /identity/v3/users/4a34d6938614463883e4b5fdfc976b49 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:25.314902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bc32-5c7c-42c3-a331-9ece9bbd0284 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:25.322582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bc32-5c7c-42c3-a331-9ece9bbd0284 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:25.323199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bc32-5c7c-42c3-a331-9ece9bbd0284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:25.386540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c940df02ebff4e94bf0874df846db424', 'request_id': 'req-18cbf06a-b403-45dc-89c9-fb03d1769f55'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.398827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7be8bc32-5c7c-42c3-a331-9ece9bbd0284 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:25.400310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7be8bc32-5c7c-42c3-a331-9ece9bbd0284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:25.401951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c940df02ebff4e94bf0874df846db424', 'request_id': 'req-18cbf06a-b403-45dc-89c9-fb03d1769f55'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.421597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c940df02ebff4e94bf0874df846db424', 'request_id': 'req-18cbf06a-b403-45dc-89c9-fb03d1769f55'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.447179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:25.466487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] The token cache is being invalidate because project c940df02ebff4e94bf0874df846db424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:25.466776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.467462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-18cbf06a-b403-45dc-89c9-fb03d1769f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:25.474020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 841/1688] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:25 2022] DELETE /identity/v3/projects/c940df02ebff4e94bf0874df846db424 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:25.475992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781627-76b9-4e5a-9be0-e5905243de21 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:25.476408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781627-76b9-4e5a-9be0-e5905243de21 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:25.476612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781627-76b9-4e5a-9be0-e5905243de21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:25.570597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-89781627-76b9-4e5a-9be0-e5905243de21 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:25.571879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-89781627-76b9-4e5a-9be0-e5905243de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:25.615877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 848/1689] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:25.641391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8bc32-5c7c-42c3-a331-9ece9bbd0284 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:25.642963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7be8bc32-5c7c-42c3-a331-9ece9bbd0284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:25.675141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:25.745499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 842/1690] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:25.762273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3717-901e-4499-b3a7-ba3d4ededf27 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:25.762703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3717-901e-4499-b3a7-ba3d4ededf27 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:25.763088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3717-901e-4499-b3a7-ba3d4ededf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:25.796768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8bc32-5c7c-42c3-a331-9ece9bbd0284 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:25.804012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad403669-85e1-4400-a1ca-61ec57c72a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:25.806373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad403669-85e1-4400-a1ca-61ec57c72a63 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:25.806893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad403669-85e1-4400-a1ca-61ec57c72a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:25.806893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad403669-85e1-4400-a1ca-61ec57c72a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:25.822563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-bcdf3717-901e-4499-b3a7-ba3d4ededf27 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:25.826876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-bcdf3717-901e-4499-b3a7-ba3d4ededf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:25.837074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad403669-85e1-4400-a1ca-61ec57c72a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:25.838613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad403669-85e1-4400-a1ca-61ec57c72a63 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:25.850238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 843/1691] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:25.854534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3717-901e-4499-b3a7-ba3d4ededf27 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:25.870380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 849/1692] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:20:25.889899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3717-901e-4499-b3a7-ba3d4ededf27 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:25.913431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:25.914127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:25.915876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:25.917600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:25.919711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:25.919824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:25.921555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:26.002694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 844/1693] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:20:26.010235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad403669-85e1-4400-a1ca-61ec57c72a63 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:26.027246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad403669-85e1-4400-a1ca-61ec57c72a63 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:26.043756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd628-f6ae-479d-9044-262b65ca535c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:26.044406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd628-f6ae-479d-9044-262b65ca535c admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:26.044684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd628-f6ae-479d-9044-262b65ca535c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:26.044956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd628-f6ae-479d-9044-262b65ca535c admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:26.048675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fd628-f6ae-479d-9044-262b65ca535c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:26.055321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fd628-f6ae-479d-9044-262b65ca535c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:26.167949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:26.175482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 850/1694] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:26 2022] POST /identity/v3/projects => generated 356 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:26.187309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9159517-8c05-46b2-9447-dd2976cbb803 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:26.356739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5893f-f050-4d43-b364-8a03bbd53469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:26.357576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5893f-f050-4d43-b364-8a03bbd53469 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:26.357863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5893f-f050-4d43-b364-8a03bbd53469 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:26.358150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5893f-f050-4d43-b364-8a03bbd53469 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:26.358980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5893f-f050-4d43-b364-8a03bbd53469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:26.360964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5893f-f050-4d43-b364-8a03bbd53469 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:26.404923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 845/1695] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:26 2022] POST /identity/v3/users => generated 279 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:26.408685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd628-f6ae-479d-9044-262b65ca535c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:26.426832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd628-f6ae-479d-9044-262b65ca535c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:26.440419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e639392-a055-4981-a4b6-16ffc23c0cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:26.440843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e639392-a055-4981-a4b6-16ffc23c0cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:26.441126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e639392-a055-4981-a4b6-16ffc23c0cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:26.441493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e639392-a055-4981-a4b6-16ffc23c0cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:26.442410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e639392-a055-4981-a4b6-16ffc23c0cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:26.445104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e639392-a055-4981-a4b6-16ffc23c0cb3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:26.478932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5893f-f050-4d43-b364-8a03bbd53469 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:26.498919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 851/1696] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:26 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:26.516567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5893f-f050-4d43-b364-8a03bbd53469 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:26.535640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d4769-4f71-4a76-b258-f01f661f48e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:26.536442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d4769-4f71-4a76-b258-f01f661f48e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:26.536724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d4769-4f71-4a76-b258-f01f661f48e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:26.536996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d4769-4f71-4a76-b258-f01f661f48e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:26.537744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d4769-4f71-4a76-b258-f01f661f48e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:26.539381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d4769-4f71-4a76-b258-f01f661f48e4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:26.570495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 846/1697] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:26 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:26.573665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e639392-a055-4981-a4b6-16ffc23c0cb3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:26.589772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e639392-a055-4981-a4b6-16ffc23c0cb3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:26.609064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799df3-99f5-4484-a99f-d357e3ddd829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:26.609784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799df3-99f5-4484-a99f-d357e3ddd829 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:26.610063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799df3-99f5-4484-a99f-d357e3ddd829 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:26.610336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799df3-99f5-4484-a99f-d357e3ddd829 admin admin] PATH_INFO: `/v3/projects/02f0a5e7a52841458392c66ce5c9ae9d/users/82afbcddbdb44ba09cda37d7ec157083/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:26.648400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb799df3-99f5-4484-a99f-d357e3ddd829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f0a5e7a52841458392c66ce5c9ae9d, user_id=82afbcddbdb44ba09cda37d7ec157083, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:26.651127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb799df3-99f5-4484-a99f-d357e3ddd829 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:26.675491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 852/1698] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:26 2022] PUT /identity/v3/projects/02f0a5e7a52841458392c66ce5c9ae9d/users/82afbcddbdb44ba09cda37d7ec157083/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:20:26.688007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1000-24d0-4a5c-934f-32394ef41bf2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:26.688511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1000-24d0-4a5c-934f-32394ef41bf2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:26.688751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1000-24d0-4a5c-934f-32394ef41bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:26.726580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7fdd1000-24d0-4a5c-934f-32394ef41bf2 None None] MFA Rules not processed for user `82afbcddbdb44ba09cda37d7ec157083`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:26.727919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7fdd1000-24d0-4a5c-934f-32394ef41bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:26.840973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799df3-99f5-4484-a99f-d357e3ddd829 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:26.854935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 847/1699] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:26.973399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799df3-99f5-4484-a99f-d357e3ddd829 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:26.993092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440f35b-5039-4478-b760-68b6e474bfc4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:26.993092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440f35b-5039-4478-b760-68b6e474bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:26.993092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440f35b-5039-4478-b760-68b6e474bfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:26.995135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c44ce1-7438-49cb-b88d-30f0828293f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:26.999898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c44ce1-7438-49cb-b88d-30f0828293f8 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:27.000416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c44ce1-7438-49cb-b88d-30f0828293f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:27.002731 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c44ce1-7438-49cb-b88d-30f0828293f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:27.006636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c44ce1-7438-49cb-b88d-30f0828293f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:27.017369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c44ce1-7438-49cb-b88d-30f0828293f8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:27.050924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:27.052088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0440f35b-5039-4478-b760-68b6e474bfc4 None None] MFA Rules not processed for user `f3dab07011da408c97ed90d58f2c155b`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:27.055654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0440f35b-5039-4478-b760-68b6e474bfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:27.094660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:27.099215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 853/1700] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:26 2022] POST /identity/v3/projects => generated 356 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:27.101569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c44ce1-7438-49cb-b88d-30f0828293f8 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:27.264445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 848/1701] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:27.277914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960855c-4338-429d-bd12-433caa2d6971 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:27.278205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960855c-4338-429d-bd12-433caa2d6971 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:27.278483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960855c-4338-429d-bd12-433caa2d6971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:27.321849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c44ce1-7438-49cb-b88d-30f0828293f8 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:27.333059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f960855c-4338-429d-bd12-433caa2d6971 None None] MFA Rules not processed for user `f3dab07011da408c97ed90d58f2c155b`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:27.334550 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f960855c-4338-429d-bd12-433caa2d6971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:27.336181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6bc20-9212-49a1-8bb0-9648cce56797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:27.339387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6bc20-9212-49a1-8bb0-9648cce56797 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:27.339877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6bc20-9212-49a1-8bb0-9648cce56797 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:27.340305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6bc20-9212-49a1-8bb0-9648cce56797 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:27.341880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6bc20-9212-49a1-8bb0-9648cce56797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:27.346407 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6bc20-9212-49a1-8bb0-9648cce56797 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:27.440892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 854/1702] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:27 2022] POST /identity/v3/users => generated 285 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:20:27.443796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6bc20-9212-49a1-8bb0-9648cce56797 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:27.493110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6bc20-9212-49a1-8bb0-9648cce56797 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:27.544761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd5990-a88c-4d7d-8f57-d80be893550f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:27.545574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd5990-a88c-4d7d-8f57-d80be893550f admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:27.545844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd5990-a88c-4d7d-8f57-d80be893550f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:27.546116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd5990-a88c-4d7d-8f57-d80be893550f admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:27.546878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd5990-a88c-4d7d-8f57-d80be893550f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:27.562621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd5990-a88c-4d7d-8f57-d80be893550f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:27.574760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 849/1703] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:27 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:27.575944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960855c-4338-429d-bd12-433caa2d6971 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:27.587395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f960855c-4338-429d-bd12-433caa2d6971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:27.600720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 855/1704] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:27 2022] GET /identity/v3/roles => generated 1427 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:27.601975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd5990-a88c-4d7d-8f57-d80be893550f admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:27.634530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960855c-4338-429d-bd12-433caa2d6971 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:27.646071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd5990-a88c-4d7d-8f57-d80be893550f admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:27.650686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:27.651142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:27.655428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:27.655428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] PATH_INFO: `/v3/projects/d885cc270f764ae289330459966bddbc` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:27.655705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d885cc270f764ae289330459966bddbc)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:27.659517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:27.659960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:27.661018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:27.661331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:27.661839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] PATH_INFO: `/v3/projects/b6eb4944ec594d259baeded9e5fc74bc/users/b079b9eea1d34a9f90572b64cbae32c3/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:27.712379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6eb4944ec594d259baeded9e5fc74bc, user_id=b079b9eea1d34a9f90572b64cbae32c3, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:27.715823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:27.761529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 856/1705] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:27 2022] PUT /identity/v3/projects/b6eb4944ec594d259baeded9e5fc74bc/users/b079b9eea1d34a9f90572b64cbae32c3/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:27.761529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:27.768001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:27.876988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd885cc270f764ae289330459966bddbc', 'request_id': 'req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:27.920606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd885cc270f764ae289330459966bddbc', 'request_id': 'req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:27.935823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd885cc270f764ae289330459966bddbc', 'request_id': 'req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:27.997499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] The token cache is being invalidate because project d885cc270f764ae289330459966bddbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:27.997818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.023105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.031415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 850/1706] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:27 2022] DELETE /identity/v3/projects/d885cc270f764ae289330459966bddbc => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:28.033158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:28.038688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:28.078284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21e2f5-8b9b-4b36-8dd7-ce9018db8bf6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:28.178042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:28.230075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:28.240220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8df902-7d9b-48aa-bf10-9c92e6ba44d8 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:28.266493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68ec22-05ee-4c6f-a96f-6ca074b05374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:28.267635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68ec22-05ee-4c6f-a96f-6ca074b05374 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:28.268077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68ec22-05ee-4c6f-a96f-6ca074b05374 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:28.270056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68ec22-05ee-4c6f-a96f-6ca074b05374 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:28.272170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68ec22-05ee-4c6f-a96f-6ca074b05374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:28.280660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68ec22-05ee-4c6f-a96f-6ca074b05374 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:28.304231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:28.304999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:28.307355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:28.307789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:28.314667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 851/1707] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:28 2022] GET /identity/v3/roles => generated 1427 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:28.318437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:28.321500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35954e-1b5a-4ab4-a90e-d6b559702673 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:28.321791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35954e-1b5a-4ab4-a90e-d6b559702673 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:28.322066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35954e-1b5a-4ab4-a90e-d6b559702673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:28.457612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-4b35954e-1b5a-4ab4-a90e-d6b559702673 None None] MFA Rules not processed for user `8c50160e51b54cde9a3f617d71fbf1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:28.462184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4b35954e-1b5a-4ab4-a90e-d6b559702673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:28.499339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:28.502704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:28.559678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 857/1708] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:27 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 799 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:28.560683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:28.564524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:28.617530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c084-8df9-4236-8da2-be8345fd8ba3 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:28.628023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 852/1709] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:28.629596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35954e-1b5a-4ab4-a90e-d6b559702673 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:28.633778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4b35954e-1b5a-4ab4-a90e-d6b559702673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:28.636431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:28.637109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:28.637390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:28.637699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] PATH_INFO: `/v3/users/394287cf90934789803be4f51aa37d78` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:28.641534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=394287cf90934789803be4f51aa37d78)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:28.643475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:28.673390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35954e-1b5a-4ab4-a90e-d6b559702673 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:28.688258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:28.688985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:28.689266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:28.690506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] PATH_INFO: `/v3/users/bb940b4935e54eeb9409fb02b05c5e02` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:28.693162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb940b4935e54eeb9409fb02b05c5e02)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:28.699669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:28.761859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '394287cf90934789803be4f51aa37d78'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.784134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '394287cf90934789803be4f51aa37d78'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.800755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '394287cf90934789803be4f51aa37d78'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.802043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '394287cf90934789803be4f51aa37d78'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.834619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb940b4935e54eeb9409fb02b05c5e02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.837149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '394287cf90934789803be4f51aa37d78'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.848396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb940b4935e54eeb9409fb02b05c5e02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.849767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb940b4935e54eeb9409fb02b05c5e02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.868876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 858/1710] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:28 2022] DELETE /identity/v3/users/394287cf90934789803be4f51aa37d78 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:28.869841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:28.876836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:28.899722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb940b4935e54eeb9409fb02b05c5e02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.901162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5db9c23d-44cd-4953-bbb1-fbe44382d6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb940b4935e54eeb9409fb02b05c5e02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:28.947003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 853/1711] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:28 2022] DELETE /identity/v3/users/bb940b4935e54eeb9409fb02b05c5e02 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:28.952871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4369d-e54e-40b5-96c2-0c90b7621171 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:28.955056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4369d-e54e-40b5-96c2-0c90b7621171 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:28.955580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4369d-e54e-40b5-96c2-0c90b7621171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:29.037385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:29.068742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6243054-4760-42c0-8f0d-b72d0226cfa5 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:29.167166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-99d4369d-e54e-40b5-96c2-0c90b7621171 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:29.177112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-99d4369d-e54e-40b5-96c2-0c90b7621171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:29.223294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:29.305634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:29.310455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:29.310836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:29.311179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] PATH_INFO: `/v3/projects/b6eb4944ec594d259baeded9e5fc74bc/users/b079b9eea1d34a9f90572b64cbae32c3/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:29.321601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6eb4944ec594d259baeded9e5fc74bc, user_id=b079b9eea1d34a9f90572b64cbae32c3, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:29.332414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:29.367528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 854/1712] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:20:28 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:29.372304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a68c1-958f-4017-9e9d-c087cd032c67 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:29.372586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a68c1-958f-4017-9e9d-c087cd032c67 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:29.374122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a68c1-958f-4017-9e9d-c087cd032c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:29.382863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 859/1713] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:28 2022] PUT /identity/v3/projects/b6eb4944ec594d259baeded9e5fc74bc/users/b079b9eea1d34a9f90572b64cbae32c3/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:29.384091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:29.397280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:29.484919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-1b5a68c1-958f-4017-9e9d-c087cd032c67 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:29.492476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1b5a68c1-958f-4017-9e9d-c087cd032c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:29.704757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 855/1714] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:29.706175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce5bd4-1ac2-43c1-aeef-ee9bfc690a4d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:29.706462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce5bd4-1ac2-43c1-aeef-ee9bfc690a4d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:29.706739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce5bd4-1ac2-43c1-aeef-ee9bfc690a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:29.718549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c3656-954b-4062-8f70-ac64f12aaa69 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:29.755209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:29.760158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:29.762436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:29.763078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:29.769102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:29.778068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-37ce5bd4-1ac2-43c1-aeef-ee9bfc690a4d None None] MFA Rules not processed for user `b079b9eea1d34a9f90572b64cbae32c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:29.779285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-37ce5bd4-1ac2-43c1-aeef-ee9bfc690a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:29.971544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:29.975618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:29.987016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 856/1715] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:29.998297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce5bd4-1ac2-43c1-aeef-ee9bfc690a4d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:30.003071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-37ce5bd4-1ac2-43c1-aeef-ee9bfc690a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:30.019422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 860/1716] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:30.020457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:30.024660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:30.129400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:30.186798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce5bd4-1ac2-43c1-aeef-ee9bfc690a4d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:30.191930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:30.197824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:30.198477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:30.198756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:30.200649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:30.229875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:30.232921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:30.288768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 857/1717] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:30.291511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:30.294970 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:30.376136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54581612-817d-4b36-805b-e476683cbe18 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:30.390493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de439c5-7c78-416f-bb56-163096a74e7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:30.391726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de439c5-7c78-416f-bb56-163096a74e7a service swift] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:30.393120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de439c5-7c78-416f-bb56-163096a74e7a service swift] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:30.393433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de439c5-7c78-416f-bb56-163096a74e7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:30.488931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83464b03-11fb-4aa7-a44c-eac59cea2331 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:30.503136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd8286-df46-40c1-bc66-6bcdf32bca81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:30.503775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8286-df46-40c1-bc66-6bcdf32bca81 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:30.504050 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8286-df46-40c1-bc66-6bcdf32bca81 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:30.504322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8286-df46-40c1-bc66-6bcdf32bca81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:30.527182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd8286-df46-40c1-bc66-6bcdf32bca81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:30.528958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd8286-df46-40c1-bc66-6bcdf32bca81 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:30.581380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 858/1718] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:30.583026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de439c5-7c78-416f-bb56-163096a74e7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:30.584785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de439c5-7c78-416f-bb56-163096a74e7a service swift] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:30.598153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 861/1719] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:20:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:20:30.674819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbdbe1-cc21-4f92-9e1e-1d6675b1cebd None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:30.675107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbdbe1-cc21-4f92-9e1e-1d6675b1cebd None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:30.675383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbdbe1-cc21-4f92-9e1e-1d6675b1cebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:30.703610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-c9cbdbe1-cc21-4f92-9e1e-1d6675b1cebd None None] MFA Rules not processed for user `5455c117e349480ba8b8c020b767bec5`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:30.705160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c9cbdbe1-cc21-4f92-9e1e-1d6675b1cebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:30.715879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 859/1720] 10.222.0.42 () {58 vars in 1019 bytes} [Thu Aug 4 22:20:30 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:30.761034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de439c5-7c78-416f-bb56-163096a74e7a service swift] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:30.770833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbdbe1-cc21-4f92-9e1e-1d6675b1cebd None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:30.808182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de439c5-7c78-416f-bb56-163096a74e7a service swift] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:30.809377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbdbe1-cc21-4f92-9e1e-1d6675b1cebd None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:30.823062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e258a25-a4ec-4528-b5ee-696fbb4da51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:30.824766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:30.826158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:30.827017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e258a25-a4ec-4528-b5ee-696fbb4da51f service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:30.828445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e258a25-a4ec-4528-b5ee-696fbb4da51f service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:30.829080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:30.829766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e258a25-a4ec-4528-b5ee-696fbb4da51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:30.830215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:30.844486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2e258a25-a4ec-4528-b5ee-696fbb4da51f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:30.886640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:30.888458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:30.935013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 860/1721] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:30 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:31.071049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e258a25-a4ec-4528-b5ee-696fbb4da51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:31.076204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e258a25-a4ec-4528-b5ee-696fbb4da51f service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:31.128684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 862/1722] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:32.003158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:32.011167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:32.057709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18c4d3-6133-4fe2-80a0-8fe7f8dabb9a service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:32.082467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:32.083663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:32.084196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:32.084691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] PATH_INFO: `/v3/projects/469f1d1ea08b4d2fab96d575a070d977` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:32.088358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=469f1d1ea08b4d2fab96d575a070d977)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:32.094469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:32.278680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '469f1d1ea08b4d2fab96d575a070d977', 'request_id': 'req-f95bd810-40f7-4be3-857c-f80a13bcd936'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:32.297623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '469f1d1ea08b4d2fab96d575a070d977', 'request_id': 'req-f95bd810-40f7-4be3-857c-f80a13bcd936'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:32.339854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '469f1d1ea08b4d2fab96d575a070d977', 'request_id': 'req-f95bd810-40f7-4be3-857c-f80a13bcd936'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:32.372617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] The token cache is being invalidate because project 469f1d1ea08b4d2fab96d575a070d977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:32.373626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:32.383588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f95bd810-40f7-4be3-857c-f80a13bcd936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:32.390997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 861/1723] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:32 2022] DELETE /identity/v3/projects/469f1d1ea08b4d2fab96d575a070d977 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:32.400957 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e129-9462-41ce-ade4-6497e1907278 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:32.401562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e129-9462-41ce-ade4-6497e1907278 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:32.401562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e129-9462-41ce-ade4-6497e1907278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:32.478140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-0e48e129-9462-41ce-ade4-6497e1907278 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:32.479959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0e48e129-9462-41ce-ade4-6497e1907278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:32.617965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:32.641225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 863/1724] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:32.653966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0debe6-6644-4689-aa6b-a9dc728e8593 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:32.654918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0debe6-6644-4689-aa6b-a9dc728e8593 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:32.655908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0debe6-6644-4689-aa6b-a9dc728e8593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:32.827195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2b0debe6-6644-4689-aa6b-a9dc728e8593 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:32.836924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2b0debe6-6644-4689-aa6b-a9dc728e8593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:32.880650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 862/1725] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:32.885065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e129-9462-41ce-ade4-6497e1907278 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:32.912746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e129-9462-41ce-ade4-6497e1907278 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:32.964092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa48d1d-3225-4743-9882-91402e0af023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:32.964772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa48d1d-3225-4743-9882-91402e0af023 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:32.965022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa48d1d-3225-4743-9882-91402e0af023 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:32.965293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa48d1d-3225-4743-9882-91402e0af023 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:32.978055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa48d1d-3225-4743-9882-91402e0af023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:32.978454 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa48d1d-3225-4743-9882-91402e0af023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:32.986080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa48d1d-3225-4743-9882-91402e0af023 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:33.072286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0debe6-6644-4689-aa6b-a9dc728e8593 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:33.086246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0debe6-6644-4689-aa6b-a9dc728e8593 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:33.096284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 864/1726] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:33.130334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15845cb7-151e-43b5-8845-ff6ed5c818f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:33.139560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845cb7-151e-43b5-8845-ff6ed5c818f4 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:33.139911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845cb7-151e-43b5-8845-ff6ed5c818f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:33.140255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845cb7-151e-43b5-8845-ff6ed5c818f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:33.150347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15845cb7-151e-43b5-8845-ff6ed5c818f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:33.157675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15845cb7-151e-43b5-8845-ff6ed5c818f4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:33.222818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471ed8-76bb-4123-852f-e56ca3627f1e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:33.223348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471ed8-76bb-4123-852f-e56ca3627f1e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:33.223768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471ed8-76bb-4123-852f-e56ca3627f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:33.355256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-30471ed8-76bb-4123-852f-e56ca3627f1e None None] MFA Rules not processed for user `f3dab07011da408c97ed90d58f2c155b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:33.364932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-30471ed8-76bb-4123-852f-e56ca3627f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:33.444961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 863/1727] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:33 2022] POST /identity/v3/projects => generated 362 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:20:33.449698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15845cb7-151e-43b5-8845-ff6ed5c818f4 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:33.560247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:33.661439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 865/1728] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:33 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:33.663050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30471ed8-76bb-4123-852f-e56ca3627f1e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:33.664685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-30471ed8-76bb-4123-852f-e56ca3627f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:33.685993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30471ed8-76bb-4123-852f-e56ca3627f1e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:33.819545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15845cb7-151e-43b5-8845-ff6ed5c818f4 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:33.848467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d66ec3-21b6-4736-9ade-5f064e82cd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:33.850111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d66ec3-21b6-4736-9ade-5f064e82cd46 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:33.850585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d66ec3-21b6-4736-9ade-5f064e82cd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:33.852993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d66ec3-21b6-4736-9ade-5f064e82cd46 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:33.860478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d66ec3-21b6-4736-9ade-5f064e82cd46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:33.869339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d66ec3-21b6-4736-9ade-5f064e82cd46 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:33.981040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 864/1729] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:33 2022] POST /identity/v3/users => generated 282 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:33.985106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d66ec3-21b6-4736-9ade-5f064e82cd46 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:34.000207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:34.004580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:34.004959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:34.005240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] PATH_INFO: `/v3/projects/7f5901777ea64cf995c5626bcc782115` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:34.012934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5901777ea64cf995c5626bcc782115)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:34.013912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d66ec3-21b6-4736-9ade-5f064e82cd46 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:34.025310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:34.048082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c96b5-5b35-4caf-9f0b-bd801aee85a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:34.049073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c96b5-5b35-4caf-9f0b-bd801aee85a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:34.051885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c96b5-5b35-4caf-9f0b-bd801aee85a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:34.052423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c96b5-5b35-4caf-9f0b-bd801aee85a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:34.053365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c96b5-5b35-4caf-9f0b-bd801aee85a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:34.057381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c96b5-5b35-4caf-9f0b-bd801aee85a9 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:34.114300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 865/1730] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:33 2022] GET /identity/v3/roles => generated 1427 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:34.119468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c96b5-5b35-4caf-9f0b-bd801aee85a9 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:34.148527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c96b5-5b35-4caf-9f0b-bd801aee85a9 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:34.163868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9115f-5d93-4120-a432-1ae701af3bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:34.167421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9115f-5d93-4120-a432-1ae701af3bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:34.168366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9115f-5d93-4120-a432-1ae701af3bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:34.168845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9115f-5d93-4120-a432-1ae701af3bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:34.169816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9115f-5d93-4120-a432-1ae701af3bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:34.177620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9115f-5d93-4120-a432-1ae701af3bc4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:34.205827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 866/1731] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:34 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:34.210190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9115f-5d93-4120-a432-1ae701af3bc4 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:34.244204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9115f-5d93-4120-a432-1ae701af3bc4 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:34.262094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5901777ea64cf995c5626bcc782115', 'request_id': 'req-089d0111-a247-4117-8cd9-ffcb9582e12a'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:34.272470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5901777ea64cf995c5626bcc782115', 'request_id': 'req-089d0111-a247-4117-8cd9-ffcb9582e12a'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:34.291645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5901777ea64cf995c5626bcc782115', 'request_id': 'req-089d0111-a247-4117-8cd9-ffcb9582e12a'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:34.325132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] The token cache is being invalidate because project 7f5901777ea64cf995c5626bcc782115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:34.325434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:34.327815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-089d0111-a247-4117-8cd9-ffcb9582e12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:34.340872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 866/1732] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:33 2022] DELETE /identity/v3/projects/7f5901777ea64cf995c5626bcc782115 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:34.346126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ec62-ed94-4149-b2a3-c1808b66dd2d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:34.346690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ec62-ed94-4149-b2a3-c1808b66dd2d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:34.348276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ec62-ed94-4149-b2a3-c1808b66dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:34.382458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7317ec62-ed94-4149-b2a3-c1808b66dd2d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:34.384191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7317ec62-ed94-4149-b2a3-c1808b66dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:34.392400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:34.451293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe251937-ea83-4205-abed-b00e8616c88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:34.454915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251937-ea83-4205-abed-b00e8616c88b admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:34.460821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251937-ea83-4205-abed-b00e8616c88b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:34.461362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251937-ea83-4205-abed-b00e8616c88b admin admin] PATH_INFO: `/v3/projects/ea8d3832d8b84768881484d6ec634c14/users/69ef36e73ba243ddaeefe870b58f2965/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:34.505271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 867/1733] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:34.514225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93dd09-9dbe-4a52-93d4-c16524a7b4d0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:34.514809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93dd09-9dbe-4a52-93d4-c16524a7b4d0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:34.515256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93dd09-9dbe-4a52-93d4-c16524a7b4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:34.517197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe251937-ea83-4205-abed-b00e8616c88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8d3832d8b84768881484d6ec634c14, user_id=69ef36e73ba243ddaeefe870b58f2965, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:34.529996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe251937-ea83-4205-abed-b00e8616c88b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:34.559435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 867/1734] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:34 2022] PUT /identity/v3/projects/ea8d3832d8b84768881484d6ec634c14/users/69ef36e73ba243ddaeefe870b58f2965/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:34.574544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8a93dd09-9dbe-4a52-93d4-c16524a7b4d0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:34.576607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ceed-bcf7-47fe-b821-495e9465afb6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:34.576978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ceed-bcf7-47fe-b821-495e9465afb6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:34.577339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3ceed-bcf7-47fe-b821-495e9465afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:34.599228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8a93dd09-9dbe-4a52-93d4-c16524a7b4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:34.683463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-78f3ceed-bcf7-47fe-b821-495e9465afb6 None None] MFA Rules not processed for user `69ef36e73ba243ddaeefe870b58f2965`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:34.686454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-78f3ceed-bcf7-47fe-b821-495e9465afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:34.859726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 868/1735] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:34.864741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93dd09-9dbe-4a52-93d4-c16524a7b4d0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:34.909041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93dd09-9dbe-4a52-93d4-c16524a7b4d0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:34.918174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:34.943355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:34.943700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:34.943981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:34.944258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:34.944974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:34.945273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:34.952903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:34.968706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 868/1736] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:35.014986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 869/1737] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:35.016117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3ceed-bcf7-47fe-b821-495e9465afb6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:35.038075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3ceed-bcf7-47fe-b821-495e9465afb6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:35.062316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:35.066765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:35.067109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:35.069591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:35.070485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:35.073155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:35.178412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 869/1738] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:35 2022] POST /identity/v3/projects => generated 372 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:35.182500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:35.223499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9b377-8b2b-472a-af37-cb68c5e54082 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:35.304960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:35.307483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:35.498423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e790f0-96ac-4c3a-af03-1325e9b79306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:35.499734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e790f0-96ac-4c3a-af03-1325e9b79306 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:35.501074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e790f0-96ac-4c3a-af03-1325e9b79306 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:35.502251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e790f0-96ac-4c3a-af03-1325e9b79306 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:35.503051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e790f0-96ac-4c3a-af03-1325e9b79306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:35.514186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e790f0-96ac-4c3a-af03-1325e9b79306 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:35.530806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5da9-2b95-428e-8a1c-18ec4d330150 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:35.547009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:35.548028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:35.548496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:35.549309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:35.572618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 870/1739] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:35 2022] POST /identity/v3/users => generated 287 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:35.577625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e790f0-96ac-4c3a-af03-1325e9b79306 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:35.641567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e790f0-96ac-4c3a-af03-1325e9b79306 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:35.662124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b073df-5b3d-4e21-a72d-74b600e251f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:35.662812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b073df-5b3d-4e21-a72d-74b600e251f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:35.663093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b073df-5b3d-4e21-a72d-74b600e251f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:35.663410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b073df-5b3d-4e21-a72d-74b600e251f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:35.664293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b073df-5b3d-4e21-a72d-74b600e251f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:35.679934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b073df-5b3d-4e21-a72d-74b600e251f1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:35.738406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 871/1740] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:35 2022] GET /identity/v3/roles => generated 1427 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:20:35.743701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b073df-5b3d-4e21-a72d-74b600e251f1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:35.777761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b073df-5b3d-4e21-a72d-74b600e251f1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:35.804709 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb3ac1-da19-4eb3-a4fa-5885848ddf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:35.804885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:35.808973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:35.809964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb3ac1-da19-4eb3-a4fa-5885848ddf27 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:35.811888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb3ac1-da19-4eb3-a4fa-5885848ddf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:35.812421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb3ac1-da19-4eb3-a4fa-5885848ddf27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:35.813365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb3ac1-da19-4eb3-a4fa-5885848ddf27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:35.821125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb3ac1-da19-4eb3-a4fa-5885848ddf27 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:35.840070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 870/1741] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:35.859070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:35.864946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:35.875928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 872/1742] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:35 2022] GET /identity/v3/roles => generated 1427 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:35.879577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb3ac1-da19-4eb3-a4fa-5885848ddf27 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:35.924483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf8669-3abf-41e2-b163-c4b6e9a72697 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:35.928436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb3ac1-da19-4eb3-a4fa-5885848ddf27 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:35.944543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f49aa-4c4e-4376-a916-1549f80c812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:35.946508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f49aa-4c4e-4376-a916-1549f80c812f admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:35.949761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f49aa-4c4e-4376-a916-1549f80c812f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:35.950260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f49aa-4c4e-4376-a916-1549f80c812f admin admin] PATH_INFO: `/v3/projects/f8ebecad38ce4047b46640d6e87e2455/users/2b49db2407cf4664a992f5ebc7ce8ddf/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:35.954897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:35.955515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:35.955802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:35.956078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] PATH_INFO: `/v3/users/69ef36e73ba243ddaeefe870b58f2965` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:35.958522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:35.975664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ef36e73ba243ddaeefe870b58f2965)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:35.978968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:36.041978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f49aa-4c4e-4376-a916-1549f80c812f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ebecad38ce4047b46640d6e87e2455, user_id=2b49db2407cf4664a992f5ebc7ce8ddf, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:36.053220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f49aa-4c4e-4376-a916-1549f80c812f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:36.098296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 873/1743] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:35 2022] PUT /identity/v3/projects/f8ebecad38ce4047b46640d6e87e2455/users/2b49db2407cf4664a992f5ebc7ce8ddf/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:36.099895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb1c8e-c456-437a-9c23-e2bbdeeda1ec None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:36.100186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb1c8e-c456-437a-9c23-e2bbdeeda1ec None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:36.100461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb1c8e-c456-437a-9c23-e2bbdeeda1ec None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:36.110318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 874/1744] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:20:36 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:20:36.111766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e565ae-e5a4-4bf7-98bb-06d50f6fbe1f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:36.112102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e565ae-e5a4-4bf7-98bb-06d50f6fbe1f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:36.112423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e565ae-e5a4-4bf7-98bb-06d50f6fbe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:36.119383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ef36e73ba243ddaeefe870b58f2965'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:36.140353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ef36e73ba243ddaeefe870b58f2965'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:36.141327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ef36e73ba243ddaeefe870b58f2965'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:36.187153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-14e565ae-e5a4-4bf7-98bb-06d50f6fbe1f None None] MFA Rules not processed for user `2b49db2407cf4664a992f5ebc7ce8ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:36.192860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-14e565ae-e5a4-4bf7-98bb-06d50f6fbe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:36.195995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ef36e73ba243ddaeefe870b58f2965'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:36.197619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2fae8bb9-62b6-44ae-b37d-6beb2d416a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ef36e73ba243ddaeefe870b58f2965'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:36.266975 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 871/1745] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:35 2022] DELETE /identity/v3/users/69ef36e73ba243ddaeefe870b58f2965 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:36.272775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5d973-191c-4cc5-82f9-987395a7bfaf None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:36.273083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5d973-191c-4cc5-82f9-987395a7bfaf None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:36.273370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5d973-191c-4cc5-82f9-987395a7bfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:36.313687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:36.321116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 875/1746] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:36.322987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae0df3-8f03-44e4-91e9-a233c6003e78 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:36.323391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae0df3-8f03-44e4-91e9-a233c6003e78 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:36.323895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae0df3-8f03-44e4-91e9-a233c6003e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:36.348381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-d3ae0df3-8f03-44e4-91e9-a233c6003e78 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:36.349893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d3ae0df3-8f03-44e4-91e9-a233c6003e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:36.366112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-bbd5d973-191c-4cc5-82f9-987395a7bfaf None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:36.369865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-bbd5d973-191c-4cc5-82f9-987395a7bfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:36.435799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 876/1747] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:36.436712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae0df3-8f03-44e4-91e9-a233c6003e78 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:36.438260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d3ae0df3-8f03-44e4-91e9-a233c6003e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:36.454600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae0df3-8f03-44e4-91e9-a233c6003e78 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:36.461298 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:36.462445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:36.463085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:36.463610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:36.465777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:36.468260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:36.528418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 877/1748] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:36 2022] POST /identity/v3/projects => generated 374 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:36.530778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:36.532974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:36.606942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 872/1749] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:20:36 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:36.609167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5d973-191c-4cc5-82f9-987395a7bfaf None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:36.637651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5d973-191c-4cc5-82f9-987395a7bfaf None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:36.679084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148ed4a-27e4-45fe-a386-945ebf6588cf admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:36.690197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd12b8d-037c-4f7b-acf1-ad37a8f9a971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:36.692197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd12b8d-037c-4f7b-acf1-ad37a8f9a971 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:36.692791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd12b8d-037c-4f7b-acf1-ad37a8f9a971 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:36.694539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd12b8d-037c-4f7b-acf1-ad37a8f9a971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:36.721796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:36.957259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd12b8d-037c-4f7b-acf1-ad37a8f9a971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:36.959291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94176050-d6a6-4b73-a0c3-76d9ddd730a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:36.960114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd12b8d-037c-4f7b-acf1-ad37a8f9a971 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:36.961487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94176050-d6a6-4b73-a0c3-76d9ddd730a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:36.961487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94176050-d6a6-4b73-a0c3-76d9ddd730a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:36.961487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94176050-d6a6-4b73-a0c3-76d9ddd730a5 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:36.973084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94176050-d6a6-4b73-a0c3-76d9ddd730a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:36.980462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 878/1750] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:36.991833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94176050-d6a6-4b73-a0c3-76d9ddd730a5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:37.077459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 873/1751] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:36 2022] POST /identity/v3/users => generated 294 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:37.082360 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd12b8d-037c-4f7b-acf1-ad37a8f9a971 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:37.128915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd12b8d-037c-4f7b-acf1-ad37a8f9a971 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:37.153235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7a2da-6d42-49f4-baf0-d401eee69ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:37.157306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a2da-6d42-49f4-baf0-d401eee69ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:37.158294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a2da-6d42-49f4-baf0-d401eee69ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:37.158586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a2da-6d42-49f4-baf0-d401eee69ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:37.159323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7a2da-6d42-49f4-baf0-d401eee69ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:37.169341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7a2da-6d42-49f4-baf0-d401eee69ac6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:37.210438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 879/1752] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:37 2022] GET /identity/v3/roles => generated 1427 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:37.211723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94176050-d6a6-4b73-a0c3-76d9ddd730a5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:37.231469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94176050-d6a6-4b73-a0c3-76d9ddd730a5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:37.240933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:37.242669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:37.243221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:37.243221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] PATH_INFO: `/v3/projects/c027837362c649d9b56d7260f29bb0a7/users/939a3008eefd4a7a8a7bc58d2b97e093/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:37.299943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c027837362c649d9b56d7260f29bb0a7, user_id=939a3008eefd4a7a8a7bc58d2b97e093, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:37.304686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:37.336950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 874/1753] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:37 2022] PUT /identity/v3/projects/c027837362c649d9b56d7260f29bb0a7/users/939a3008eefd4a7a8a7bc58d2b97e093/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:37.340930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7a2da-6d42-49f4-baf0-d401eee69ac6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:37.370170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7a2da-6d42-49f4-baf0-d401eee69ac6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:37.627675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc4639-ed16-47fc-987f-34d3f641a21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:37.628890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc4639-ed16-47fc-987f-34d3f641a21c admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:37.634506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc4639-ed16-47fc-987f-34d3f641a21c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:37.635457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc4639-ed16-47fc-987f-34d3f641a21c admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:37.644307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc4639-ed16-47fc-987f-34d3f641a21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:37.657873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc4639-ed16-47fc-987f-34d3f641a21c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:37.664638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:37.666860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:37.754499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 880/1754] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:37 2022] GET /identity/v3/roles => generated 1427 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:37.755575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc4639-ed16-47fc-987f-34d3f641a21c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:37.796047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc4639-ed16-47fc-987f-34d3f641a21c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:37.827378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb388f18-2d7a-4d8b-9d4b-5be0508527d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:37.830514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb388f18-2d7a-4d8b-9d4b-5be0508527d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:37.830805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb388f18-2d7a-4d8b-9d4b-5be0508527d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:37.831089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb388f18-2d7a-4d8b-9d4b-5be0508527d0 admin admin] PATH_INFO: `/v3/projects/c027837362c649d9b56d7260f29bb0a7/users/939a3008eefd4a7a8a7bc58d2b97e093/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:37.847284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb388f18-2d7a-4d8b-9d4b-5be0508527d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c027837362c649d9b56d7260f29bb0a7, user_id=939a3008eefd4a7a8a7bc58d2b97e093, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:37.865214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb388f18-2d7a-4d8b-9d4b-5be0508527d0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:37.923655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 881/1755] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:37 2022] PUT /identity/v3/projects/c027837362c649d9b56d7260f29bb0a7/users/939a3008eefd4a7a8a7bc58d2b97e093/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:37.926013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:37.935647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e133d-474a-4383-aa3c-0fff3bf8c7f4 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:37.935936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e133d-474a-4383-aa3c-0fff3bf8c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:37.936213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e133d-474a-4383-aa3c-0fff3bf8c7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:37.987150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a3464-fcac-4cc5-892f-9dabeba3e8b0 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:38.082056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a65e133d-474a-4383-aa3c-0fff3bf8c7f4 None None] MFA Rules not processed for user `939a3008eefd4a7a8a7bc58d2b97e093`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:38.091479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a65e133d-474a-4383-aa3c-0fff3bf8c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:38.250508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:38.251186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:38.251467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:38.251687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:38.290362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:38.291768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:38.337340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 875/1756] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:37 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:20:38.394513 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:38.419773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 882/1757] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:38.537868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:38.539512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:38.791717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9540-b984-4170-9244-5a3c0b58d1eb service nova] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:38.824056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21f515-646c-4b54-bea5-783d45a9c8f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:38.824710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21f515-646c-4b54-bea5-783d45a9c8f0 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:38.824990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21f515-646c-4b54-bea5-783d45a9c8f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:38.825260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21f515-646c-4b54-bea5-783d45a9c8f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:38.867843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21f515-646c-4b54-bea5-783d45a9c8f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:38.869303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21f515-646c-4b54-bea5-783d45a9c8f0 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:38.904863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 876/1758] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:39.829304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e133d-474a-4383-aa3c-0fff3bf8c7f4 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:40.105775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e133d-474a-4383-aa3c-0fff3bf8c7f4 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:40.124912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:40.126086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:40.126585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:40.126902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] PATH_INFO: `/v3/users/2b49db2407cf4664a992f5ebc7ce8ddf` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:40.129742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b49db2407cf4664a992f5ebc7ce8ddf)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:40.132139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:40.319804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b49db2407cf4664a992f5ebc7ce8ddf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:40.365293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b49db2407cf4664a992f5ebc7ce8ddf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:40.401781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b49db2407cf4664a992f5ebc7ce8ddf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:40.402891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b49db2407cf4664a992f5ebc7ce8ddf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:40.425502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d7b955f1-4445-4c56-8437-f7858639e431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b49db2407cf4664a992f5ebc7ce8ddf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:40.446506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 883/1759] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:39 2022] DELETE /identity/v3/users/2b49db2407cf4664a992f5ebc7ce8ddf => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:40.452923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208218-e2d0-4e8e-92ac-8f360d24b1c2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:40.454109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208218-e2d0-4e8e-92ac-8f360d24b1c2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:40.455641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208218-e2d0-4e8e-92ac-8f360d24b1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:40.515879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-cf208218-e2d0-4e8e-92ac-8f360d24b1c2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:40.518158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-cf208218-e2d0-4e8e-92ac-8f360d24b1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:40.557568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5f2d-7cf4-49c9-9410-fb45bdd20c1d None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:40.560091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5f2d-7cf4-49c9-9410-fb45bdd20c1d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:40.560694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a5f2d-7cf4-49c9-9410-fb45bdd20c1d None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:40.562956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 884/1760] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:20:40 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:20:40.635772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f79c1-b48d-4f25-98ea-3d6877f67f2e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:40.636155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f79c1-b48d-4f25-98ea-3d6877f67f2e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:40.636584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f79c1-b48d-4f25-98ea-3d6877f67f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:40.720649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6e2f79c1-b48d-4f25-98ea-3d6877f67f2e None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:40.727486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6e2f79c1-b48d-4f25-98ea-3d6877f67f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:40.767355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 877/1761] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:41.076699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 885/1762] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:20:40 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:41.327267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf208218-e2d0-4e8e-92ac-8f360d24b1c2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:41.329722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-cf208218-e2d0-4e8e-92ac-8f360d24b1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:41.407676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:41.470928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf208218-e2d0-4e8e-92ac-8f360d24b1c2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:41.491362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:41.492096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:41.492354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:41.492625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:41.511070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:41.511855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:41.537014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 878/1763] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:41.778045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f79c1-b48d-4f25-98ea-3d6877f67f2e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:41.780150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6e2f79c1-b48d-4f25-98ea-3d6877f67f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:41.824450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f79c1-b48d-4f25-98ea-3d6877f67f2e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:41.856111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf730ba-27c6-4c9c-b68d-65af5c6ad292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:41.858283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf730ba-27c6-4c9c-b68d-65af5c6ad292 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:41.859989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf730ba-27c6-4c9c-b68d-65af5c6ad292 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:41.860570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf730ba-27c6-4c9c-b68d-65af5c6ad292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:41.883167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:41.892559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf730ba-27c6-4c9c-b68d-65af5c6ad292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:41.895526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf730ba-27c6-4c9c-b68d-65af5c6ad292 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:41.943771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 886/1764] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:41 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:20:42.860386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:42.862020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:42.908841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9589d5-71a9-46a4-b029-cb2be6efef9b service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:42.926111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:42.927395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:42.927747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:42.928185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] PATH_INFO: `/v3/projects/ea8d3832d8b84768881484d6ec634c14` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:42.931011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8d3832d8b84768881484d6ec634c14)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:42.933912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:43.091572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8d3832d8b84768881484d6ec634c14', 'request_id': 'req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:43.109197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8d3832d8b84768881484d6ec634c14', 'request_id': 'req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:43.126583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8d3832d8b84768881484d6ec634c14', 'request_id': 'req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:43.156232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] The token cache is being invalidate because project ea8d3832d8b84768881484d6ec634c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:43.156786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:43.158172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37c8cd97-1f00-4866-b1bc-ffa6f299ac38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:43.163997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 879/1765] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:42 2022] DELETE /identity/v3/projects/ea8d3832d8b84768881484d6ec634c14 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:43.165976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863c49b-1ba9-4a4e-b8cb-67875504ff78 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:43.166491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863c49b-1ba9-4a4e-b8cb-67875504ff78 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:43.166835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863c49b-1ba9-4a4e-b8cb-67875504ff78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:43.196021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-d863c49b-1ba9-4a4e-b8cb-67875504ff78 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:43.197621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d863c49b-1ba9-4a4e-b8cb-67875504ff78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:43.291014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 887/1766] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:43.302537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4302d-ce94-4829-8825-e187aee7bcd0 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:43.304750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4302d-ce94-4829-8825-e187aee7bcd0 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:43.308334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4302d-ce94-4829-8825-e187aee7bcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:43.363180 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-46e4302d-ce94-4829-8825-e187aee7bcd0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:43.371706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-46e4302d-ce94-4829-8825-e187aee7bcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:43.398557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 880/1767] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:43.402763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863c49b-1ba9-4a4e-b8cb-67875504ff78 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:43.419516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863c49b-1ba9-4a4e-b8cb-67875504ff78 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:43.456964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:43.458215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:43.458794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:43.461390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:43.463099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:43.463790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:43.479555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:43.529829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 888/1768] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:20:43.542517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4302d-ce94-4829-8825-e187aee7bcd0 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:43.564342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4302d-ce94-4829-8825-e187aee7bcd0 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:43.575879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178c36c-3c91-4c3b-8927-9ca55499dfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:43.576558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178c36c-3c91-4c3b-8927-9ca55499dfa2 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:43.576953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178c36c-3c91-4c3b-8927-9ca55499dfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:43.577227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178c36c-3c91-4c3b-8927-9ca55499dfa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:43.578104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178c36c-3c91-4c3b-8927-9ca55499dfa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:43.580984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178c36c-3c91-4c3b-8927-9ca55499dfa2 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:43.702158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:43.715248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 881/1769] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:43 2022] POST /identity/v3/projects => generated 374 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:43.719480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a31bd-fa00-4cd9-ba79-49c471652fd9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:43.894606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fb2dd-8d9e-4730-8ba8-14845dfbc943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:43.895901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb2dd-8d9e-4730-8ba8-14845dfbc943 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:43.896390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb2dd-8d9e-4730-8ba8-14845dfbc943 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:43.900479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb2dd-8d9e-4730-8ba8-14845dfbc943 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:43.901599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fb2dd-8d9e-4730-8ba8-14845dfbc943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:43.905053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fb2dd-8d9e-4730-8ba8-14845dfbc943 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:43.945533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178c36c-3c91-4c3b-8927-9ca55499dfa2 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:43.956883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 889/1770] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:43 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:20:43.963387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178c36c-3c91-4c3b-8927-9ca55499dfa2 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:43.968986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48260-52c4-4785-b51c-da32065838aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:43.970266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48260-52c4-4785-b51c-da32065838aa admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:43.970870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48260-52c4-4785-b51c-da32065838aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:43.971337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48260-52c4-4785-b51c-da32065838aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:43.972723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e48260-52c4-4785-b51c-da32065838aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:43.975302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e48260-52c4-4785-b51c-da32065838aa admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:43.999631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fb2dd-8d9e-4730-8ba8-14845dfbc943 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:44.006390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 882/1771] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:43 2022] GET /identity/v3/roles => generated 1427 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:44.011331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fb2dd-8d9e-4730-8ba8-14845dfbc943 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:44.016320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:44.016876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:44.017366 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:44.017837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:44.019403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:44.022003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:44.040986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48260-52c4-4785-b51c-da32065838aa admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:44.042192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 890/1772] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:43 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:44.063636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48260-52c4-4785-b51c-da32065838aa admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:44.071327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d503ba6-fa9e-4861-9e77-738a1ac14c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:44.072339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d503ba6-fa9e-4861-9e77-738a1ac14c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:44.073023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d503ba6-fa9e-4861-9e77-738a1ac14c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:44.078002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:44.079702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d503ba6-fa9e-4861-9e77-738a1ac14c25 admin admin] PATH_INFO: `/v3/projects/dca74d8db43048189703565e6ba4e465/users/b54f3553528c4e4c974f7f4d4f9668bb/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:44.082139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:44.110834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7ea2a-daf4-4bff-b213-595bca392a42 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:44.129507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:44.130562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:44.131054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:44.131711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] PATH_INFO: `/v3/projects/f8ebecad38ce4047b46640d6e87e2455` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:44.134959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ebecad38ce4047b46640d6e87e2455)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:44.139560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d503ba6-fa9e-4861-9e77-738a1ac14c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca74d8db43048189703565e6ba4e465, user_id=b54f3553528c4e4c974f7f4d4f9668bb, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:44.142778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:44.146957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d503ba6-fa9e-4861-9e77-738a1ac14c25 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:44.175935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 883/1773] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:44 2022] PUT /identity/v3/projects/dca74d8db43048189703565e6ba4e465/users/b54f3553528c4e4c974f7f4d4f9668bb/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:44.192238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e8e89-2b35-4099-95e3-7e1cfeb0449e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:44.193471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e8e89-2b35-4099-95e3-7e1cfeb0449e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:44.194006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e8e89-2b35-4099-95e3-7e1cfeb0449e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:44.282770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-296e8e89-2b35-4099-95e3-7e1cfeb0449e None None] MFA Rules not processed for user `b54f3553528c4e4c974f7f4d4f9668bb`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:44.284767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-296e8e89-2b35-4099-95e3-7e1cfeb0449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:44.310213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ebecad38ce4047b46640d6e87e2455', 'request_id': 'req-604750fc-c274-4842-ad25-a5af03054a79'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:44.343667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:44.347958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ebecad38ce4047b46640d6e87e2455', 'request_id': 'req-604750fc-c274-4842-ad25-a5af03054a79'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:44.377214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ebecad38ce4047b46640d6e87e2455', 'request_id': 'req-604750fc-c274-4842-ad25-a5af03054a79'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:44.399606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:44.457857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 884/1774] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:44.464151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] The token cache is being invalidate because project f8ebecad38ce4047b46640d6e87e2455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:44.467539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:44.471891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e8e89-2b35-4099-95e3-7e1cfeb0449e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:44.473891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-296e8e89-2b35-4099-95e3-7e1cfeb0449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:44.475682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:44.488144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 891/1775] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:44 2022] DELETE /identity/v3/projects/f8ebecad38ce4047b46640d6e87e2455 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:44.490548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:44.497960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:44.644223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e8e89-2b35-4099-95e3-7e1cfeb0449e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:44.656508 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe9f19-4c42-4949-9181-550bd79c3b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:44.657678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9f19-4c42-4949-9181-550bd79c3b57 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:44.658240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9f19-4c42-4949-9181-550bd79c3b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:44.658761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9f19-4c42-4949-9181-550bd79c3b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:44.662005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-73fe9f19-4c42-4949-9181-550bd79c3b57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:44.693822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe9f19-4c42-4949-9181-550bd79c3b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:44.696235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe9f19-4c42-4949-9181-550bd79c3b57 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:44.728442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 885/1776] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:44.734560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604750fc-c274-4842-ad25-a5af03054a79 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:44.752317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:44.754223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:44.754531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:44.754809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] PATH_INFO: `/v3/users/939a3008eefd4a7a8a7bc58d2b97e093` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:44.756120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=939a3008eefd4a7a8a7bc58d2b97e093)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:44.758094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:44.886858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939a3008eefd4a7a8a7bc58d2b97e093'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:44.904547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b773c3-647e-4e05-8964-fedb1575c406 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:44.904985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b773c3-647e-4e05-8964-fedb1575c406 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:44.905345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b773c3-647e-4e05-8964-fedb1575c406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:44.922877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939a3008eefd4a7a8a7bc58d2b97e093'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:44.947480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-25b773c3-647e-4e05-8964-fedb1575c406 None None] MFA Rules not processed for user `8c50160e51b54cde9a3f617d71fbf1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:44.948999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939a3008eefd4a7a8a7bc58d2b97e093'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:44.949615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-25b773c3-647e-4e05-8964-fedb1575c406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:44.952465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939a3008eefd4a7a8a7bc58d2b97e093'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:45.008835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939a3008eefd4a7a8a7bc58d2b97e093'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:45.029178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 892/1777] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:20:44 2022] DELETE /identity/v3/users/939a3008eefd4a7a8a7bc58d2b97e093 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:45.151041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 886/1778] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:45.942182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:45.943672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:46.042401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcc2a8-79e1-4e92-a9d9-a1ba36d94c25 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:46.047619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:46.048297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.048735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.049019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] PATH_INFO: `/v3/projects/c027837362c649d9b56d7260f29bb0a7` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.050942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c027837362c649d9b56d7260f29bb0a7)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:46.052735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:46.069649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:46.118563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c027837362c649d9b56d7260f29bb0a7', 'request_id': 'req-7e73f7b1-f271-443f-b14b-85294783f1b9'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:46.127428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c027837362c649d9b56d7260f29bb0a7', 'request_id': 'req-7e73f7b1-f271-443f-b14b-85294783f1b9'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:46.137075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c027837362c649d9b56d7260f29bb0a7', 'request_id': 'req-7e73f7b1-f271-443f-b14b-85294783f1b9'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:46.151011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] The token cache is being invalidate because project c027837362c649d9b56d7260f29bb0a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:20:46.151318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:46.152060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e73f7b1-f271-443f-b14b-85294783f1b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:20:46.153623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 893/1779] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:20:45 2022] DELETE /identity/v3/projects/c027837362c649d9b56d7260f29bb0a7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:46.158616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52d49-db79-4f71-9e13-14c19a8421fa None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.160366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52d49-db79-4f71-9e13-14c19a8421fa None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.160553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52d49-db79-4f71-9e13-14c19a8421fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.186654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8cd52d49-db79-4f71-9e13-14c19a8421fa None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:46.187889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8cd52d49-db79-4f71-9e13-14c19a8421fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:46.262086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 887/1780] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:20:46.270396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065eaf6-61d6-474c-9eb6-72866e177681 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.270800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065eaf6-61d6-474c-9eb6-72866e177681 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.271389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065eaf6-61d6-474c-9eb6-72866e177681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.306532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a065eaf6-61d6-474c-9eb6-72866e177681 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:46.307658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a065eaf6-61d6-474c-9eb6-72866e177681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:46.316932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 894/1781] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:46.321402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd52d49-db79-4f71-9e13-14c19a8421fa None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:46.334548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd52d49-db79-4f71-9e13-14c19a8421fa None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:46.342776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:46.344284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.345085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.345611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.347222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:20:46.348440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:46.350606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:46.373558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 888/1782] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:20:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:20:46.378438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065eaf6-61d6-474c-9eb6-72866e177681 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:46.397332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065eaf6-61d6-474c-9eb6-72866e177681 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:46.402158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e5e63-6bea-44fe-b7ee-5b49969d4a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:46.402785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e5e63-6bea-44fe-b7ee-5b49969d4a34 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.403064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e5e63-6bea-44fe-b7ee-5b49969d4a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.403408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e5e63-6bea-44fe-b7ee-5b49969d4a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.404076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e5e63-6bea-44fe-b7ee-5b49969d4a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:46.406053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e5e63-6bea-44fe-b7ee-5b49969d4a34 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:46.451037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 895/1783] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:46 2022] POST /identity/v3/projects => generated 402 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:46.455196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:46.467445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b2cc8-f467-4360-9924-4fcc09a7793e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:46.483193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:46.557877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8d3c5-426b-4e89-8d5b-3596f815dfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:46.558733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8d3c5-426b-4e89-8d5b-3596f815dfc8 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.559167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8d3c5-426b-4e89-8d5b-3596f815dfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.559677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8d3c5-426b-4e89-8d5b-3596f815dfc8 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.560769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8d3c5-426b-4e89-8d5b-3596f815dfc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:46.562979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8d3c5-426b-4e89-8d5b-3596f815dfc8 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:46.607409 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 889/1784] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:46 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:46.608671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e5e63-6bea-44fe-b7ee-5b49969d4a34 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:46.621172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e5e63-6bea-44fe-b7ee-5b49969d4a34 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:46.628638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848c75-558f-498b-baf7-8af08a206494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:46.629282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848c75-558f-498b-baf7-8af08a206494 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.629616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848c75-558f-498b-baf7-8af08a206494 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.629889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848c75-558f-498b-baf7-8af08a206494 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.630648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76848c75-558f-498b-baf7-8af08a206494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:46.637746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76848c75-558f-498b-baf7-8af08a206494 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:46.657496 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 896/1785] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:46 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:46.662146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8d3c5-426b-4e89-8d5b-3596f815dfc8 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:46.677141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8d3c5-426b-4e89-8d5b-3596f815dfc8 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:46.682527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed08ad8-bf4f-4a22-872b-b4a895b0a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:46.683099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed08ad8-bf4f-4a22-872b-b4a895b0a1ed admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.683383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed08ad8-bf4f-4a22-872b-b4a895b0a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.683690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed08ad8-bf4f-4a22-872b-b4a895b0a1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.684396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed08ad8-bf4f-4a22-872b-b4a895b0a1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:46.686034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed08ad8-bf4f-4a22-872b-b4a895b0a1ed admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:46.698440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 890/1786] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:46 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:20:46.702912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848c75-558f-498b-baf7-8af08a206494 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:46.716905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848c75-558f-498b-baf7-8af08a206494 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:46.722811 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89de237-d4f9-4815-9a63-24e92ab69466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:46.723567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89de237-d4f9-4815-9a63-24e92ab69466 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.723851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89de237-d4f9-4815-9a63-24e92ab69466 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.724130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89de237-d4f9-4815-9a63-24e92ab69466 admin admin] PATH_INFO: `/v3/projects/5ac081f5dfd841e99cdf577f34fd9d1d/users/a06edd255ee445e4bc9477f90e8348b3/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.743163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89de237-d4f9-4815-9a63-24e92ab69466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac081f5dfd841e99cdf577f34fd9d1d, user_id=a06edd255ee445e4bc9477f90e8348b3, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:46.746007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89de237-d4f9-4815-9a63-24e92ab69466 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:46.762028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 897/1787] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:46 2022] PUT /identity/v3/projects/5ac081f5dfd841e99cdf577f34fd9d1d/users/a06edd255ee445e4bc9477f90e8348b3/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:46.774516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a0229-a159-40a2-9836-0af8d08fd832 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.774804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a0229-a159-40a2-9836-0af8d08fd832 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.775393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a0229-a159-40a2-9836-0af8d08fd832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.821676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-fa9a0229-a159-40a2-9836-0af8d08fd832 None None] MFA Rules not processed for user `a06edd255ee445e4bc9477f90e8348b3`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:46.822884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fa9a0229-a159-40a2-9836-0af8d08fd832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:46.892219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 891/1788] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:46 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:46.897014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89de237-d4f9-4815-9a63-24e92ab69466 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:46.965915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89de237-d4f9-4815-9a63-24e92ab69466 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:46.970627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082ae3-03dd-4aa4-a788-5caeb4850d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:46.971843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082ae3-03dd-4aa4-a788-5caeb4850d03 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:46.972577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082ae3-03dd-4aa4-a788-5caeb4850d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:46.972577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082ae3-03dd-4aa4-a788-5caeb4850d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:46.973368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b082ae3-03dd-4aa4-a788-5caeb4850d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:46.975308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b082ae3-03dd-4aa4-a788-5caeb4850d03 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:47.003721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 898/1789] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:20:46 2022] POST /identity/v3/projects => generated 400 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:47.007758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a0229-a159-40a2-9836-0af8d08fd832 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:47.018727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a0229-a159-40a2-9836-0af8d08fd832 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:47.082874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf27483-1a53-43d3-9d9d-3f7570350eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:47.083513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf27483-1a53-43d3-9d9d-3f7570350eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:47.083791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf27483-1a53-43d3-9d9d-3f7570350eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:47.084266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf27483-1a53-43d3-9d9d-3f7570350eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:47.084741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf27483-1a53-43d3-9d9d-3f7570350eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:47.086783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf27483-1a53-43d3-9d9d-3f7570350eb9 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:47.116473 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 892/1790] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:20:47 2022] POST /identity/v3/users => generated 307 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:20:47.122534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082ae3-03dd-4aa4-a788-5caeb4850d03 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:47.134369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082ae3-03dd-4aa4-a788-5caeb4850d03 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:47.139046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff29e40-afe1-413d-af2e-559f14d23009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:47.139662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff29e40-afe1-413d-af2e-559f14d23009 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:47.139940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff29e40-afe1-413d-af2e-559f14d23009 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:47.140208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff29e40-afe1-413d-af2e-559f14d23009 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:47.140893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff29e40-afe1-413d-af2e-559f14d23009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:47.142505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff29e40-afe1-413d-af2e-559f14d23009 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:47.156644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 899/1791] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:47 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:47.161568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf27483-1a53-43d3-9d9d-3f7570350eb9 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:47.171411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf27483-1a53-43d3-9d9d-3f7570350eb9 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:47.176211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbdd77-816a-4906-9a73-7e1e9f5dd7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:47.176873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbdd77-816a-4906-9a73-7e1e9f5dd7f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:47.177152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbdd77-816a-4906-9a73-7e1e9f5dd7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:47.177446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbdd77-816a-4906-9a73-7e1e9f5dd7f2 admin admin] PATH_INFO: `/v3/projects/9bb8e641e95244298342afe4c781e25f/users/ab7f6e1458724fdd84662e63071e9dcc/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:47.185857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:47.193608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbdd77-816a-4906-9a73-7e1e9f5dd7f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb8e641e95244298342afe4c781e25f, user_id=ab7f6e1458724fdd84662e63071e9dcc, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:47.196570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbdd77-816a-4906-9a73-7e1e9f5dd7f2 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:47.211205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 893/1792] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:47 2022] PUT /identity/v3/projects/9bb8e641e95244298342afe4c781e25f/users/ab7f6e1458724fdd84662e63071e9dcc/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:20:47.215144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff29e40-afe1-413d-af2e-559f14d23009 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:47.222258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:47.226466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff29e40-afe1-413d-af2e-559f14d23009 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:47.288169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52616bf3-accc-4a6d-961e-ad8703206bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:47.288796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52616bf3-accc-4a6d-961e-ad8703206bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:47.289080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52616bf3-accc-4a6d-961e-ad8703206bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:47.289354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52616bf3-accc-4a6d-961e-ad8703206bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:47.290171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52616bf3-accc-4a6d-961e-ad8703206bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:47.291870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52616bf3-accc-4a6d-961e-ad8703206bd6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:47.302882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 900/1793] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:20:47 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:20:47.309352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbdd77-816a-4906-9a73-7e1e9f5dd7f2 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:47.319745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbdd77-816a-4906-9a73-7e1e9f5dd7f2 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:47.324396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:47.325056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:47.325542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:47.325819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] PATH_INFO: `/v3/projects/9bb8e641e95244298342afe4c781e25f/users/ab7f6e1458724fdd84662e63071e9dcc/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:47.327909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb8e641e95244298342afe4c781e25f, user_id=ab7f6e1458724fdd84662e63071e9dcc, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:47.331479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:47.342609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 894/1794] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:20:47 2022] PUT /identity/v3/projects/9bb8e641e95244298342afe4c781e25f/users/ab7f6e1458724fdd84662e63071e9dcc/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:20:47.353798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ea51-1cbb-4d67-9826-a6cf5b6762a4 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:47.354092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ea51-1cbb-4d67-9826-a6cf5b6762a4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:47.354374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ea51-1cbb-4d67-9826-a6cf5b6762a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:47.395199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5f37ea51-1cbb-4d67-9826-a6cf5b6762a4 None None] MFA Rules not processed for user `ab7f6e1458724fdd84662e63071e9dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:20:47.396715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5f37ea51-1cbb-4d67-9826-a6cf5b6762a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:47.525856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 901/1795] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:20:47 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:20:47.535336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:47.537277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:47.602562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52982c7-5a28-41e3-bfb1-b54fa6be6602 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:47.606504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08288073-451b-4e4b-9b99-12ffadb8221e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:47.607127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08288073-451b-4e4b-9b99-12ffadb8221e service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:47.607404 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08288073-451b-4e4b-9b99-12ffadb8221e service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:47.607674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08288073-451b-4e4b-9b99-12ffadb8221e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:47.618034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08288073-451b-4e4b-9b99-12ffadb8221e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:47.619490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08288073-451b-4e4b-9b99-12ffadb8221e service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:47.631352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 895/1796] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:47 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:48.008968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37ea51-1cbb-4d67-9826-a6cf5b6762a4 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:20:48.010500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5f37ea51-1cbb-4d67-9826-a6cf5b6762a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:20:48.031367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37ea51-1cbb-4d67-9826-a6cf5b6762a4 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:20:48.040605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ad03c-dd28-46ff-8026-4fc9b1afda9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:20:48.041702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ad03c-dd28-46ff-8026-4fc9b1afda9d service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:20:48.042269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ad03c-dd28-46ff-8026-4fc9b1afda9d service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:20:48.043005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ad03c-dd28-46ff-8026-4fc9b1afda9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:20:48.115543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ad03c-dd28-46ff-8026-4fc9b1afda9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:20:48.117142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ad03c-dd28-46ff-8026-4fc9b1afda9d service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:20:48.126618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 902/1797] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:20:48 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:20:57.625563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=36,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:20:58.122080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:07.825310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247bea3-10bb-4bc5-a21a-c142a26fc144 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:07.825998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247bea3-10bb-4bc5-a21a-c142a26fc144 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:07.826525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247bea3-10bb-4bc5-a21a-c142a26fc144 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:07.828605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 896/1798] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:21:07 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:21:07.837816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2bc5d-5206-42ad-8a27-dea5daedc031 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:07.838389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2bc5d-5206-42ad-8a27-dea5daedc031 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:07.838748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2bc5d-5206-42ad-8a27-dea5daedc031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:07.869233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-31e2bc5d-5206-42ad-8a27-dea5daedc031 None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:07.870824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-31e2bc5d-5206-42ad-8a27-dea5daedc031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:07.997010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 903/1799] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:21:07 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:08.029126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247bea3-10bb-4bc5-a21a-c142a26fc144 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:08.031062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5247bea3-10bb-4bc5-a21a-c142a26fc144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:08.061679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247bea3-10bb-4bc5-a21a-c142a26fc144 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:08.076537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb5095-78e1-413f-ac60-ce2881be4ad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:08.077730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb5095-78e1-413f-ac60-ce2881be4ad6 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:08.078124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb5095-78e1-413f-ac60-ce2881be4ad6 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:08.078712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb5095-78e1-413f-ac60-ce2881be4ad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:08.091351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb5095-78e1-413f-ac60-ce2881be4ad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:08.095482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb5095-78e1-413f-ac60-ce2881be4ad6 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:08.110724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 897/1800] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:08 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:08.230252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2bc5d-5206-42ad-8a27-dea5daedc031 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:08.309752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2bc5d-5206-42ad-8a27-dea5daedc031 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:08.314342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:08.314985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:08.315261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:08.315530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] PATH_INFO: `/v3/users/a06edd255ee445e4bc9477f90e8348b3` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:08.316864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06edd255ee445e4bc9477f90e8348b3)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:08.318818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:08.375125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06edd255ee445e4bc9477f90e8348b3'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.383873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06edd255ee445e4bc9477f90e8348b3'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.391646 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06edd255ee445e4bc9477f90e8348b3'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.392280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06edd255ee445e4bc9477f90e8348b3'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.407753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06edd255ee445e4bc9477f90e8348b3'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.414518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 904/1801] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:08 2022] DELETE /identity/v3/users/a06edd255ee445e4bc9477f90e8348b3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:08.418965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c71a9-27cd-4910-bd24-4dd8f7e65009 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:08.419307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c71a9-27cd-4910-bd24-4dd8f7e65009 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:08.419604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c71a9-27cd-4910-bd24-4dd8f7e65009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:08.438284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e71c71a9-27cd-4910-bd24-4dd8f7e65009 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:08.439488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e71c71a9-27cd-4910-bd24-4dd8f7e65009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:08.470322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:08.472029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:08.487411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:08.507785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 898/1802] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:08.517355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c71a9-27cd-4910-bd24-4dd8f7e65009 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:08.518800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e71c71a9-27cd-4910-bd24-4dd8f7e65009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:08.555614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb7287-78c7-45a0-b375-1ab19e2616df admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:08.560728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:08.561600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:08.562151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:08.562502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] PATH_INFO: `/v3/users/82afbcddbdb44ba09cda37d7ec157083` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:08.563762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82afbcddbdb44ba09cda37d7ec157083)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:08.566636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:08.591000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c71a9-27cd-4910-bd24-4dd8f7e65009 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:08.596374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c111d4-539c-4dff-9c66-4525c5a416c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:08.597051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c111d4-539c-4dff-9c66-4525c5a416c3 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:08.597448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c111d4-539c-4dff-9c66-4525c5a416c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:08.597753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c111d4-539c-4dff-9c66-4525c5a416c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:08.611049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c111d4-539c-4dff-9c66-4525c5a416c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:08.613688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c111d4-539c-4dff-9c66-4525c5a416c3 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:08.623230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82afbcddbdb44ba09cda37d7ec157083'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.626061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 899/1803] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:21:08.637129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82afbcddbdb44ba09cda37d7ec157083'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.648999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82afbcddbdb44ba09cda37d7ec157083'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.650323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82afbcddbdb44ba09cda37d7ec157083'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.669508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82afbcddbdb44ba09cda37d7ec157083'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:08.681457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 905/1804] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:08 2022] DELETE /identity/v3/users/82afbcddbdb44ba09cda37d7ec157083 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:08.683461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cc03a-581a-4b0c-a265-c1f1489c1433 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:08.683949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cc03a-581a-4b0c-a265-c1f1489c1433 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:08.684404 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cc03a-581a-4b0c-a265-c1f1489c1433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:08.710023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ae2cc03a-581a-4b0c-a265-c1f1489c1433 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:08.711492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ae2cc03a-581a-4b0c-a265-c1f1489c1433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:08.783698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 900/1805] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:08.796763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:08.796763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:08.883596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff785c-8d09-439c-acbe-2d39795bc2c7 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:08.889048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25529-ff89-4d22-ae45-41e52de0b6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:08.890054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25529-ff89-4d22-ae45-41e52de0b6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:08.890463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25529-ff89-4d22-ae45-41e52de0b6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:08.890819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25529-ff89-4d22-ae45-41e52de0b6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:08.902127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba25529-ff89-4d22-ae45-41e52de0b6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:08.904295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba25529-ff89-4d22-ae45-41e52de0b6b8 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:08.917710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 906/1806] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:09.150235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cc03a-581a-4b0c-a265-c1f1489c1433 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:09.151982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ae2cc03a-581a-4b0c-a265-c1f1489c1433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:09.169390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cc03a-581a-4b0c-a265-c1f1489c1433 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:09.180482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:09.181695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:09.182259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:09.182752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] PATH_INFO: `/v3/projects/5ac081f5dfd841e99cdf577f34fd9d1d` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:09.184133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac081f5dfd841e99cdf577f34fd9d1d)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:09.186374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:09.247671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:09.274221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac081f5dfd841e99cdf577f34fd9d1d', 'request_id': 'req-f79899e4-2deb-4ecf-91d2-96905ed2996e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.286923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac081f5dfd841e99cdf577f34fd9d1d', 'request_id': 'req-f79899e4-2deb-4ecf-91d2-96905ed2996e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.294653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac081f5dfd841e99cdf577f34fd9d1d', 'request_id': 'req-f79899e4-2deb-4ecf-91d2-96905ed2996e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.309074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] The token cache is being invalidate because project 5ac081f5dfd841e99cdf577f34fd9d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:09.309433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.310308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.312591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 901/1807] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:09 2022] DELETE /identity/v3/projects/5ac081f5dfd841e99cdf577f34fd9d1d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:09.316622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25529-ff89-4d22-ae45-41e52de0b6b8 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:09.349588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25529-ff89-4d22-ae45-41e52de0b6b8 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:09.431248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:09.490143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:09.491079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:09.491465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:09.491959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] PATH_INFO: `/v3/users/ab7f6e1458724fdd84662e63071e9dcc` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:09.494010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7f6e1458724fdd84662e63071e9dcc)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:09.497433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:09.596074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7f6e1458724fdd84662e63071e9dcc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.614339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7f6e1458724fdd84662e63071e9dcc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.630804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7f6e1458724fdd84662e63071e9dcc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.631617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7f6e1458724fdd84662e63071e9dcc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.657580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7f6e1458724fdd84662e63071e9dcc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:09.688875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 907/1808] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:09 2022] DELETE /identity/v3/users/ab7f6e1458724fdd84662e63071e9dcc => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:10.906746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:10.911338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:10.922340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:10.925428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:10.935062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79899e4-2deb-4ecf-91d2-96905ed2996e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:11.031598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16a5ce-6fd0-4940-8400-ee083956bfd0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:11.045502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:11.049181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:11.049502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:11.052518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] PATH_INFO: `/v3/projects/9bb8e641e95244298342afe4c781e25f` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:11.056031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb8e641e95244298342afe4c781e25f)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:11.059111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:11.103391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:11.104083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:11.104453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:11.104628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] PATH_INFO: `/v3/projects/02f0a5e7a52841458392c66ce5c9ae9d` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:11.110189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f0a5e7a52841458392c66ce5c9ae9d)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:11.114873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:11.220733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb8e641e95244298342afe4c781e25f', 'request_id': 'req-66e99f90-7efb-40fb-b932-c72dbd1ef106'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.233482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:11.234623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f0a5e7a52841458392c66ce5c9ae9d', 'request_id': 'req-bd21c020-4f79-429f-88c2-150e5e345251'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.237050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb8e641e95244298342afe4c781e25f', 'request_id': 'req-66e99f90-7efb-40fb-b932-c72dbd1ef106'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.249057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f0a5e7a52841458392c66ce5c9ae9d', 'request_id': 'req-bd21c020-4f79-429f-88c2-150e5e345251'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.260749 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb8e641e95244298342afe4c781e25f', 'request_id': 'req-66e99f90-7efb-40fb-b932-c72dbd1ef106'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.274251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f0a5e7a52841458392c66ce5c9ae9d', 'request_id': 'req-bd21c020-4f79-429f-88c2-150e5e345251'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.286899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] The token cache is being invalidate because project 9bb8e641e95244298342afe4c781e25f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:11.287480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.290072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-66e99f90-7efb-40fb-b932-c72dbd1ef106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.291557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 908/1809] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:10 2022] DELETE /identity/v3/projects/9bb8e641e95244298342afe4c781e25f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:11.298916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d64c8-ae33-4230-bf54-bd0859eb924d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:11.299466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d64c8-ae33-4230-bf54-bd0859eb924d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:11.299993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d64c8-ae33-4230-bf54-bd0859eb924d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:11.314624 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] The token cache is being invalidate because project 02f0a5e7a52841458392c66ce5c9ae9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:11.314925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.315888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.328848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 902/1810] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:10 2022] DELETE /identity/v3/projects/02f0a5e7a52841458392c66ce5c9ae9d => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:11.337385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:11.344198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:11.344377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-079d64c8-ae33-4230-bf54-bd0859eb924d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:11.348358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-079d64c8-ae33-4230-bf54-bd0859eb924d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:11.488983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:11.537174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 909/1811] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:11.540555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c010633-1a48-4a03-9b53-556fd73f4123 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:11.544017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c010633-1a48-4a03-9b53-556fd73f4123 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:11.544305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c010633-1a48-4a03-9b53-556fd73f4123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:11.559411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21c020-4f79-429f-88c2-150e5e345251 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:11.565571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:11.567463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:11.568101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:11.568627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] PATH_INFO: `/v3/users/b079b9eea1d34a9f90572b64cbae32c3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:11.575747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b079b9eea1d34a9f90572b64cbae32c3)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:11.577900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:11.580355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-1c010633-1a48-4a03-9b53-556fd73f4123 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:11.582607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-1c010633-1a48-4a03-9b53-556fd73f4123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:11.610045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 910/1812] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:11.616906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c010633-1a48-4a03-9b53-556fd73f4123 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:11.647108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c010633-1a48-4a03-9b53-556fd73f4123 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:11.656969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:11.658109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:11.658823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:11.659130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:11.659967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:11.660390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:11.664610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:11.692454 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 911/1813] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:11.694108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:11.700711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b079b9eea1d34a9f90572b64cbae32c3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.707131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8f25d-26b5-4c8b-bdff-4c14156c339e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:11.711425 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b079b9eea1d34a9f90572b64cbae32c3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.712671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2406a3a-9e7d-49cf-aac6-5f5f7b1260aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:11.712969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b079b9eea1d34a9f90572b64cbae32c3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.714381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2406a3a-9e7d-49cf-aac6-5f5f7b1260aa admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:11.714594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2406a3a-9e7d-49cf-aac6-5f5f7b1260aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:11.714936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2406a3a-9e7d-49cf-aac6-5f5f7b1260aa admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:11.715846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2406a3a-9e7d-49cf-aac6-5f5f7b1260aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:11.717968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2406a3a-9e7d-49cf-aac6-5f5f7b1260aa admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:11.735018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b079b9eea1d34a9f90572b64cbae32c3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.736018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b079b9eea1d34a9f90572b64cbae32c3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:11.751103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 903/1814] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:11 2022] DELETE /identity/v3/users/b079b9eea1d34a9f90572b64cbae32c3 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:11.800195 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 912/1815] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:11 2022] POST /identity/v3/projects => generated 386 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:11.804609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:11.806921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:11.913451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dc5ba-6c14-4d63-be59-ae92d0f8cff5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:11.980681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248752a-99f0-419d-974b-454f3e3b55c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:11.981352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248752a-99f0-419d-974b-454f3e3b55c3 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:11.982090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248752a-99f0-419d-974b-454f3e3b55c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:11.982369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248752a-99f0-419d-974b-454f3e3b55c3 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:11.983287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248752a-99f0-419d-974b-454f3e3b55c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:11.985777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248752a-99f0-419d-974b-454f3e3b55c3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.011758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:12.028634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2406a3a-9e7d-49cf-aac6-5f5f7b1260aa admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.029473 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 904/1816] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:11 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:12.043933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2406a3a-9e7d-49cf-aac6-5f5f7b1260aa admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.049346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95b13b-fd42-40e0-8b7f-51d577e81986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.050597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b13b-fd42-40e0-8b7f-51d577e81986 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.050827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b13b-fd42-40e0-8b7f-51d577e81986 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.051239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b13b-fd42-40e0-8b7f-51d577e81986 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.053587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95b13b-fd42-40e0-8b7f-51d577e81986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.055834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95b13b-fd42-40e0-8b7f-51d577e81986 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.068602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 913/1817] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:12 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:21:12.072924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248752a-99f0-419d-974b-454f3e3b55c3 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.084320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248752a-99f0-419d-974b-454f3e3b55c3 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.089321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.089992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.090293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.090778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.091623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.093708 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.118334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 905/1818] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:12 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:12.121008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95b13b-fd42-40e0-8b7f-51d577e81986 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.136413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95b13b-fd42-40e0-8b7f-51d577e81986 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.141809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fdf0b-d103-41f4-af29-2bd0842b35ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.142265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fdf0b-d103-41f4-af29-2bd0842b35ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.142541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fdf0b-d103-41f4-af29-2bd0842b35ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.142812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fdf0b-d103-41f4-af29-2bd0842b35ac admin admin] PATH_INFO: `/v3/projects/f9082cd027184f858a0507fb6688b05d/users/8cbece7efe6244b9bdfeb28110db5683/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.158835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5fdf0b-d103-41f4-af29-2bd0842b35ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9082cd027184f858a0507fb6688b05d, user_id=8cbece7efe6244b9bdfeb28110db5683, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.161878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5fdf0b-d103-41f4-af29-2bd0842b35ac admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.177358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 914/1819] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:12 2022] PUT /identity/v3/projects/f9082cd027184f858a0507fb6688b05d/users/8cbece7efe6244b9bdfeb28110db5683/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:12.183644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.185573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:12.192403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4521-58fd-4421-b1ee-28c67380b139 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.193567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4521-58fd-4421-b1ee-28c67380b139 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.196662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4521-58fd-4421-b1ee-28c67380b139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.201161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cde92-572d-4f05-817b-35db8d6840f7 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.258603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-13fa4521-58fd-4421-b1ee-28c67380b139 None None] MFA Rules not processed for user `8cbece7efe6244b9bdfeb28110db5683`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:12.260060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-13fa4521-58fd-4421-b1ee-28c67380b139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:12.302734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.304015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.304679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.305049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] PATH_INFO: `/v3/projects/b6eb4944ec594d259baeded9e5fc74bc` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.307199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6eb4944ec594d259baeded9e5fc74bc)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.309422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.331701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:12.347334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 915/1820] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:12 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:12.349271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa4521-58fd-4421-b1ee-28c67380b139 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.366454 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa4521-58fd-4421-b1ee-28c67380b139 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.367760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6eb4944ec594d259baeded9e5fc74bc', 'request_id': 'req-4b3be306-74df-410f-b13e-1c4fe5783442'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:12.377198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6eb4944ec594d259baeded9e5fc74bc', 'request_id': 'req-4b3be306-74df-410f-b13e-1c4fe5783442'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:12.384038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6eb4944ec594d259baeded9e5fc74bc', 'request_id': 'req-4b3be306-74df-410f-b13e-1c4fe5783442'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:12.397682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] The token cache is being invalidate because project b6eb4944ec594d259baeded9e5fc74bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:12.398265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:12.399332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-4b3be306-74df-410f-b13e-1c4fe5783442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:12.401294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 906/1821] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:12 2022] DELETE /identity/v3/projects/b6eb4944ec594d259baeded9e5fc74bc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:12.405783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d26e3-81d7-4057-9a14-03e0ac6cda74 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.406069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d26e3-81d7-4057-9a14-03e0ac6cda74 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.406351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d26e3-81d7-4057-9a14-03e0ac6cda74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.427675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-aa2d26e3-81d7-4057-9a14-03e0ac6cda74 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:12.428954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-aa2d26e3-81d7-4057-9a14-03e0ac6cda74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:12.430884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.431805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.431805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.432053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.432726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.434678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.437452 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 907/1822] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:12.444710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc7fb2-2da9-4e06-86d1-0f32bedd44b7 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.444999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc7fb2-2da9-4e06-86d1-0f32bedd44b7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.445275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc7fb2-2da9-4e06-86d1-0f32bedd44b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.466000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-76cc7fb2-2da9-4e06-86d1-0f32bedd44b7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:12.467257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-76cc7fb2-2da9-4e06-86d1-0f32bedd44b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:12.477520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 916/1823] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:12 2022] POST /identity/v3/projects => generated 386 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:12.478330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.479941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:12.484578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 908/1824] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:12.485581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc7fb2-2da9-4e06-86d1-0f32bedd44b7 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.498020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc7fb2-2da9-4e06-86d1-0f32bedd44b7 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.553966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4f79c-557d-4160-b045-d145a587e49d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.558775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1c5b6-2a07-4e91-b0a0-b4cf99f6f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.559331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1c5b6-2a07-4e91-b0a0-b4cf99f6f36d admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.559714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1c5b6-2a07-4e91-b0a0-b4cf99f6f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.559868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1c5b6-2a07-4e91-b0a0-b4cf99f6f36d admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.560535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1c5b6-2a07-4e91-b0a0-b4cf99f6f36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.562569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1c5b6-2a07-4e91-b0a0-b4cf99f6f36d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.580770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.582151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.582495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.582935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.583830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:12.584217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.585886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.594355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 917/1825] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:12 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:12.595768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:12.598441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1c5b6-2a07-4e91-b0a0-b4cf99f6f36d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.601569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 909/1826] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:12.604318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.610096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1c5b6-2a07-4e91-b0a0-b4cf99f6f36d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.617334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337e5fd-3684-4e4d-984d-29c62dacabf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.618079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e5fd-3684-4e4d-984d-29c62dacabf1 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.618281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e5fd-3684-4e4d-984d-29c62dacabf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.618443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661222-a4e7-4cd0-846c-4bda3f805cfb admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.618555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e5fd-3684-4e4d-984d-29c62dacabf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.619302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337e5fd-3684-4e4d-984d-29c62dacabf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.620898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337e5fd-3684-4e4d-984d-29c62dacabf1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.624037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4590ba-ca4e-44ac-8bfe-1fb08451e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.624658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4590ba-ca4e-44ac-8bfe-1fb08451e841 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.624941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4590ba-ca4e-44ac-8bfe-1fb08451e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.625220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4590ba-ca4e-44ac-8bfe-1fb08451e841 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.626064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4590ba-ca4e-44ac-8bfe-1fb08451e841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.627981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4590ba-ca4e-44ac-8bfe-1fb08451e841 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.634405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 918/1827] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:12 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:12.638548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337e5fd-3684-4e4d-984d-29c62dacabf1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.653097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337e5fd-3684-4e4d-984d-29c62dacabf1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.665012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f814b-8815-436c-bd6a-01385aa0a29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.666254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f814b-8815-436c-bd6a-01385aa0a29f admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.666918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f814b-8815-436c-bd6a-01385aa0a29f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.667378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f814b-8815-436c-bd6a-01385aa0a29f admin admin] PATH_INFO: `/v3/projects/f59b2fc0282a4d2facf1143b4ee16c30/users/d94574421e004f8ebf9f60749bea3259/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.668915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 910/1828] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:12 2022] POST /identity/v3/projects => generated 364 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:21:12.669826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4590ba-ca4e-44ac-8bfe-1fb08451e841 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.691940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f814b-8815-436c-bd6a-01385aa0a29f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59b2fc0282a4d2facf1143b4ee16c30, user_id=d94574421e004f8ebf9f60749bea3259, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.697258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f814b-8815-436c-bd6a-01385aa0a29f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.717460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 919/1829] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:12 2022] PUT /identity/v3/projects/f59b2fc0282a4d2facf1143b4ee16c30/users/d94574421e004f8ebf9f60749bea3259/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:12.721206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f814b-8815-436c-bd6a-01385aa0a29f admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.797058 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:12.811873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4590ba-ca4e-44ac-8bfe-1fb08451e841 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.827562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f814b-8815-436c-bd6a-01385aa0a29f admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.832543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54bb39-c1a4-4e79-8fb5-f2c824c94caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.833307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bb39-c1a4-4e79-8fb5-f2c824c94caa admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.841624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bb39-c1a4-4e79-8fb5-f2c824c94caa admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.841624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bb39-c1a4-4e79-8fb5-f2c824c94caa admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.841624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54bb39-c1a4-4e79-8fb5-f2c824c94caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.841624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54bb39-c1a4-4e79-8fb5-f2c824c94caa admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.854044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 920/1830] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:12 2022] GET /identity/v3/roles => generated 1427 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:12.861451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54bb39-c1a4-4e79-8fb5-f2c824c94caa admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.883774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54bb39-c1a4-4e79-8fb5-f2c824c94caa admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:12.889130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cc939-eda5-4982-b73a-e740f817ec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.889963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc939-eda5-4982-b73a-e740f817ec9d admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.890230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc939-eda5-4982-b73a-e740f817ec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.890535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc939-eda5-4982-b73a-e740f817ec9d admin admin] PATH_INFO: `/v3/projects/f59b2fc0282a4d2facf1143b4ee16c30/users/d94574421e004f8ebf9f60749bea3259/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.892542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cc939-eda5-4982-b73a-e740f817ec9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59b2fc0282a4d2facf1143b4ee16c30, user_id=d94574421e004f8ebf9f60749bea3259, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.895353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cc939-eda5-4982-b73a-e740f817ec9d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.907434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 921/1831] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:12 2022] PUT /identity/v3/projects/f59b2fc0282a4d2facf1143b4ee16c30/users/d94574421e004f8ebf9f60749bea3259/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:12.917578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3eb1a-2727-4c60-96b4-a7d992ab8ba7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.917861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3eb1a-2727-4c60-96b4-a7d992ab8ba7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.918133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3eb1a-2727-4c60-96b4-a7d992ab8ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.924318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0f46d-fe10-4f8a-8489-c7035d759043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:12.925485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0f46d-fe10-4f8a-8489-c7035d759043 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:12.925769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0f46d-fe10-4f8a-8489-c7035d759043 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:12.926100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0f46d-fe10-4f8a-8489-c7035d759043 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:12.927215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0f46d-fe10-4f8a-8489-c7035d759043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:12.930653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0f46d-fe10-4f8a-8489-c7035d759043 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:12.954210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-b2f3eb1a-2727-4c60-96b4-a7d992ab8ba7 None None] MFA Rules not processed for user `d94574421e004f8ebf9f60749bea3259`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:12.955557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b2f3eb1a-2727-4c60-96b4-a7d992ab8ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:12.961833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 911/1832] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:12 2022] POST /identity/v3/users => generated 283 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:12.966228 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0f46d-fe10-4f8a-8489-c7035d759043 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:12.979177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0f46d-fe10-4f8a-8489-c7035d759043 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.058809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e0b95-3c59-4d68-8da3-9ecdf5ef7e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.059473 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e0b95-3c59-4d68-8da3-9ecdf5ef7e6b admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.059827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e0b95-3c59-4d68-8da3-9ecdf5ef7e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.060181 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e0b95-3c59-4d68-8da3-9ecdf5ef7e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.060916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e0b95-3c59-4d68-8da3-9ecdf5ef7e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.063188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e0b95-3c59-4d68-8da3-9ecdf5ef7e6b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.064704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 922/1833] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:12 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:13.076518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3eb1a-2727-4c60-96b4-a7d992ab8ba7 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.078135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b2f3eb1a-2727-4c60-96b4-a7d992ab8ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:13.080013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 912/1834] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:12 2022] GET /identity/v3/roles => generated 1427 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:13.085185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e0b95-3c59-4d68-8da3-9ecdf5ef7e6b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.095688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e0b95-3c59-4d68-8da3-9ecdf5ef7e6b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.111774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d53ba2-ed74-41ca-9872-84b8cab9fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.113044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53ba2-ed74-41ca-9872-84b8cab9fc24 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.113245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53ba2-ed74-41ca-9872-84b8cab9fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.113568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53ba2-ed74-41ca-9872-84b8cab9fc24 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.114266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d53ba2-ed74-41ca-9872-84b8cab9fc24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.116030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d53ba2-ed74-41ca-9872-84b8cab9fc24 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.122352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:13.131050 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 913/1835] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:13 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:21:13.147453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d53ba2-ed74-41ca-9872-84b8cab9fc24 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.161798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d53ba2-ed74-41ca-9872-84b8cab9fc24 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.167818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd44dcc-4c4d-4066-9336-4f35b0b8855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.168421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3eb1a-2727-4c60-96b4-a7d992ab8ba7 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.168609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd44dcc-4c4d-4066-9336-4f35b0b8855d admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.168840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd44dcc-4c4d-4066-9336-4f35b0b8855d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.169119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd44dcc-4c4d-4066-9336-4f35b0b8855d admin admin] PATH_INFO: `/v3/projects/733f3be9ee6d45e795c8ef063b58d765/users/0822c70d5166473aa74f8522b05fbad3/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.172393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b30561-3120-4604-9894-0643643a363f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.173108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b30561-3120-4604-9894-0643643a363f service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.173399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b30561-3120-4604-9894-0643643a363f service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.173791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b30561-3120-4604-9894-0643643a363f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.176082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-78b30561-3120-4604-9894-0643643a363f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:13.189062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd44dcc-4c4d-4066-9336-4f35b0b8855d admin admin] RBAC: Authorizing `identity:create_grant(project_id=733f3be9ee6d45e795c8ef063b58d765, user_id=0822c70d5166473aa74f8522b05fbad3, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.193461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd44dcc-4c4d-4066-9336-4f35b0b8855d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.215108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 914/1836] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:13 2022] PUT /identity/v3/projects/733f3be9ee6d45e795c8ef063b58d765/users/0822c70d5166473aa74f8522b05fbad3/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:13.227589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32d54a-02f5-458b-99ff-3c3ee8e2c4fc None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.227889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32d54a-02f5-458b-99ff-3c3ee8e2c4fc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.228152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32d54a-02f5-458b-99ff-3c3ee8e2c4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.261137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b30561-3120-4604-9894-0643643a363f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.262532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b30561-3120-4604-9894-0643643a363f service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.266465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:13.272562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-dd32d54a-02f5-458b-99ff-3c3ee8e2c4fc None None] MFA Rules not processed for user `0822c70d5166473aa74f8522b05fbad3`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:13.273841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-dd32d54a-02f5-458b-99ff-3c3ee8e2c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:13.276015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 923/1837] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:13 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:21:13.366662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 915/1838] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:13.371366 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b30561-3120-4604-9894-0643643a363f service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.383236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b30561-3120-4604-9894-0643643a363f service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.464778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04df9c-c401-4644-8fa1-b878c911e68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.465558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04df9c-c401-4644-8fa1-b878c911e68a admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.465901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04df9c-c401-4644-8fa1-b878c911e68a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.466240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04df9c-c401-4644-8fa1-b878c911e68a admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.467008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04df9c-c401-4644-8fa1-b878c911e68a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.469064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04df9c-c401-4644-8fa1-b878c911e68a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.503969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 924/1839] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:13 2022] POST /identity/v3/projects => generated 364 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:21:13.509674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32d54a-02f5-458b-99ff-3c3ee8e2c4fc None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.523382 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32d54a-02f5-458b-99ff-3c3ee8e2c4fc None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.586945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abc42e-d91d-41f9-882b-78dbd07cbc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.587529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abc42e-d91d-41f9-882b-78dbd07cbc39 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.587858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abc42e-d91d-41f9-882b-78dbd07cbc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.588105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abc42e-d91d-41f9-882b-78dbd07cbc39 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.588749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abc42e-d91d-41f9-882b-78dbd07cbc39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.591019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abc42e-d91d-41f9-882b-78dbd07cbc39 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.618652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 916/1840] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:13 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:13.622606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04df9c-c401-4644-8fa1-b878c911e68a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.646181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04df9c-c401-4644-8fa1-b878c911e68a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.651828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95553b-5b92-4be9-bbd9-98cb9b98b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.652762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95553b-5b92-4be9-bbd9-98cb9b98b4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.653113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95553b-5b92-4be9-bbd9-98cb9b98b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.653488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95553b-5b92-4be9-bbd9-98cb9b98b4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.655690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95553b-5b92-4be9-bbd9-98cb9b98b4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.657301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95553b-5b92-4be9-bbd9-98cb9b98b4f1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.670917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 925/1841] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:13 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:13.675165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abc42e-d91d-41f9-882b-78dbd07cbc39 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.686139 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abc42e-d91d-41f9-882b-78dbd07cbc39 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.692862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5cb6-2588-4cb4-85d1-052cac661a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.693908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5cb6-2588-4cb4-85d1-052cac661a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.694253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5cb6-2588-4cb4-85d1-052cac661a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.694593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5cb6-2588-4cb4-85d1-052cac661a0c admin admin] PATH_INFO: `/v3/projects/cbd06b32fbdd43d4b2efd5529f4d2b14/users/caa47ad67bbe42ecaa36e469926feb47/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.711671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e5cb6-2588-4cb4-85d1-052cac661a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd06b32fbdd43d4b2efd5529f4d2b14, user_id=caa47ad67bbe42ecaa36e469926feb47, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.714720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e5cb6-2588-4cb4-85d1-052cac661a0c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.728391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 917/1842] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:13 2022] PUT /identity/v3/projects/cbd06b32fbdd43d4b2efd5529f4d2b14/users/caa47ad67bbe42ecaa36e469926feb47/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:13.732419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95553b-5b92-4be9-bbd9-98cb9b98b4f1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.742312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95553b-5b92-4be9-bbd9-98cb9b98b4f1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.799493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef78a5c-d21f-4046-946e-f5953270673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.800298 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef78a5c-d21f-4046-946e-f5953270673d admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.800575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef78a5c-d21f-4046-946e-f5953270673d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.800853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef78a5c-d21f-4046-946e-f5953270673d admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.801553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef78a5c-d21f-4046-946e-f5953270673d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.803151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef78a5c-d21f-4046-946e-f5953270673d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.816809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 926/1843] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:13 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:13.818549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5cb6-2588-4cb4-85d1-052cac661a0c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.828915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5cb6-2588-4cb4-85d1-052cac661a0c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:13.834338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:13.835009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.835293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.835609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] PATH_INFO: `/v3/projects/cbd06b32fbdd43d4b2efd5529f4d2b14/users/caa47ad67bbe42ecaa36e469926feb47/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.837657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd06b32fbdd43d4b2efd5529f4d2b14, user_id=caa47ad67bbe42ecaa36e469926feb47, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:13.840274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:13.851257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 918/1844] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:13 2022] PUT /identity/v3/projects/cbd06b32fbdd43d4b2efd5529f4d2b14/users/caa47ad67bbe42ecaa36e469926feb47/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:13.861229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce9e6-910a-475b-bb23-774c4f438f44 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:13.861554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce9e6-910a-475b-bb23-774c4f438f44 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:13.861884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce9e6-910a-475b-bb23-774c4f438f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:13.895755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-73dce9e6-910a-475b-bb23-774c4f438f44 None None] MFA Rules not processed for user `caa47ad67bbe42ecaa36e469926feb47`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:13.897083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-73dce9e6-910a-475b-bb23-774c4f438f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:13.966339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:13.982962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 927/1845] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:13.994992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:13.996891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:14.020389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:14.060727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45255dd4-4bfe-4285-99a8-7541680e1d7c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:14.064408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21815-3f1c-448b-8127-1e19bb2d6f8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:14.065029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21815-3f1c-448b-8127-1e19bb2d6f8c service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:14.065308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21815-3f1c-448b-8127-1e19bb2d6f8c service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:14.066247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21815-3f1c-448b-8127-1e19bb2d6f8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:14.075512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd21815-3f1c-448b-8127-1e19bb2d6f8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:14.076949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd21815-3f1c-448b-8127-1e19bb2d6f8c service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:14.087729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 919/1846] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:14.169431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dce9e6-910a-475b-bb23-774c4f438f44 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:14.180150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dce9e6-910a-475b-bb23-774c4f438f44 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:14.184002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834997e-06df-4f22-a690-df169611295d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:14.184615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834997e-06df-4f22-a690-df169611295d service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:14.184888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834997e-06df-4f22-a690-df169611295d service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:14.185153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834997e-06df-4f22-a690-df169611295d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:14.249733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834997e-06df-4f22-a690-df169611295d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:14.251169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834997e-06df-4f22-a690-df169611295d service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:14.260920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 928/1847] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:24.083297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=18 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:24.256696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=31 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:30.477916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21815-3f1c-448b-8127-1e19bb2d6f8c service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:30.488955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21815-3f1c-448b-8127-1e19bb2d6f8c service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:30.499371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:30.499983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:30.500261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:30.500531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:30.565791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:30.567257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:30.577762 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 920/1848] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:30 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:32.448229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834997e-06df-4f22-a690-df169611295d service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:32.450607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a834997e-06df-4f22-a690-df169611295d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:32.549819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834997e-06df-4f22-a690-df169611295d service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:32.556002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:32.556782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:32.557725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:32.558323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] PATH_INFO: `/v3/users/c4513148610f4bdaa6793fbeca2feb04` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:32.560003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4513148610f4bdaa6793fbeca2feb04)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:32.566263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:32.588519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:32.590086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:32.606042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7f2a-f1f8-43e9-b829-defcfea920c2 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:32.612767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:32.613538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:32.613821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:32.614089 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] PATH_INFO: `/v3/users/b54f3553528c4e4c974f7f4d4f9668bb` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:32.615297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54f3553528c4e4c974f7f4d4f9668bb)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:32.617231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:32.625562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4513148610f4bdaa6793fbeca2feb04'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.633905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4513148610f4bdaa6793fbeca2feb04'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.646244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4513148610f4bdaa6793fbeca2feb04'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.646953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4513148610f4bdaa6793fbeca2feb04'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.671540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c18151c4-daae-4faa-b29d-7456da8e8699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4513148610f4bdaa6793fbeca2feb04'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.684692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 929/1849] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:32 2022] DELETE /identity/v3/users/c4513148610f4bdaa6793fbeca2feb04 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:32.684836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54f3553528c4e4c974f7f4d4f9668bb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.686234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592ce7d-9d0b-4aae-9633-391d4db3a1ca None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:32.686589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592ce7d-9d0b-4aae-9633-391d4db3a1ca None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:32.686934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592ce7d-9d0b-4aae-9633-391d4db3a1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:32.693273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54f3553528c4e4c974f7f4d4f9668bb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.694174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54f3553528c4e4c974f7f4d4f9668bb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.711242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2592ce7d-9d0b-4aae-9633-391d4db3a1ca None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:32.712677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2592ce7d-9d0b-4aae-9633-391d4db3a1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:32.717987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54f3553528c4e4c974f7f4d4f9668bb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.719056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6dcaae40-d37e-460f-a5f2-aeaa1b51f599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54f3553528c4e4c974f7f4d4f9668bb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:32.762911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 921/1850] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:32 2022] DELETE /identity/v3/users/b54f3553528c4e4c974f7f4d4f9668bb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:32.765548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165888eb-6140-4d13-969b-7a1c7a8c342d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:32.766205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165888eb-6140-4d13-969b-7a1c7a8c342d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:32.766694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165888eb-6140-4d13-969b-7a1c7a8c342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:32.792516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-165888eb-6140-4d13-969b-7a1c7a8c342d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:32.793795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 930/1851] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:32.794356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-165888eb-6140-4d13-969b-7a1c7a8c342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:32.851025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592ce7d-9d0b-4aae-9633-391d4db3a1ca None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:32.854850 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2592ce7d-9d0b-4aae-9633-391d4db3a1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:32.875177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 922/1852] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:32.883818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165888eb-6140-4d13-969b-7a1c7a8c342d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:32.885307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:32.887634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-165888eb-6140-4d13-969b-7a1c7a8c342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:32.926158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:32.951696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592ce7d-9d0b-4aae-9633-391d4db3a1ca None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:33.010230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165888eb-6140-4d13-969b-7a1c7a8c342d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:33.016880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:33.018710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:33.021026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:33.021727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:33.040288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:33.042552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:33.050291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629689a-edf2-4c56-995f-41e3fcd3fcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:33.051230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629689a-edf2-4c56-995f-41e3fcd3fcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:33.051687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629689a-edf2-4c56-995f-41e3fcd3fcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:33.051968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629689a-edf2-4c56-995f-41e3fcd3fcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:33.071131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 923/1853] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:33.078283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629689a-edf2-4c56-995f-41e3fcd3fcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:33.079747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629689a-edf2-4c56-995f-41e3fcd3fcf9 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:33.109781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 931/1854] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:21:33.734133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:33.737126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:33.756184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1908f-5ab0-4e55-8ba4-c150910ba49d service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:33.761091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:33.761794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:33.762077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:33.762348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] PATH_INFO: `/v3/projects/dca74d8db43048189703565e6ba4e465` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:33.763532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca74d8db43048189703565e6ba4e465)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:33.766690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:33.817356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca74d8db43048189703565e6ba4e465', 'request_id': 'req-dfc70117-bccc-4a60-92f1-a5167c8a2324'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:33.826005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca74d8db43048189703565e6ba4e465', 'request_id': 'req-dfc70117-bccc-4a60-92f1-a5167c8a2324'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:33.834842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca74d8db43048189703565e6ba4e465', 'request_id': 'req-dfc70117-bccc-4a60-92f1-a5167c8a2324'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:33.847491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] The token cache is being invalidate because project dca74d8db43048189703565e6ba4e465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:33.847786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:33.848456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfc70117-bccc-4a60-92f1-a5167c8a2324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:33.853613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 924/1855] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:33 2022] DELETE /identity/v3/projects/dca74d8db43048189703565e6ba4e465 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:33.855308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7da1-8e75-4749-bfaa-1f7219ab8a19 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:33.856526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7da1-8e75-4749-bfaa-1f7219ab8a19 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:33.856526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7da1-8e75-4749-bfaa-1f7219ab8a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:33.878232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-966e7da1-8e75-4749-bfaa-1f7219ab8a19 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:33.879448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-966e7da1-8e75-4749-bfaa-1f7219ab8a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:33.961520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 932/1856] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:33.971330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053375-0c6c-4caf-be1e-26c55f936d7e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:33.972191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053375-0c6c-4caf-be1e-26c55f936d7e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:33.972700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053375-0c6c-4caf-be1e-26c55f936d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.030062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-17053375-0c6c-4caf-be1e-26c55f936d7e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:34.044446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-17053375-0c6c-4caf-be1e-26c55f936d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:34.067283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e7da1-8e75-4749-bfaa-1f7219ab8a19 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.067827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 925/1857] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:34.071859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053375-0c6c-4caf-be1e-26c55f936d7e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.073037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-966e7da1-8e75-4749-bfaa-1f7219ab8a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:34.098959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053375-0c6c-4caf-be1e-26c55f936d7e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.113294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e7da1-8e75-4749-bfaa-1f7219ab8a19 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.119632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.120277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.120733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.121156 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.131804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.133033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.134533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:34.134774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.135851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.137559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] PATH_INFO: `/v3/projects/042f54b443c14c41a280db13076b1fdc` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.141261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=042f54b443c14c41a280db13076b1fdc)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.142806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:34.153572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:34.179337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 926/1858] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:34.184468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.210034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ea278-eff6-47ce-8c1c-352aa03daca8 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.218884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a091f9-6b94-482a-9630-5159dcaebc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.220262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a091f9-6b94-482a-9630-5159dcaebc00 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.221797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a091f9-6b94-482a-9630-5159dcaebc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.222453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a091f9-6b94-482a-9630-5159dcaebc00 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.223451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a091f9-6b94-482a-9630-5159dcaebc00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.226074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a091f9-6b94-482a-9630-5159dcaebc00 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:34.243052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042f54b443c14c41a280db13076b1fdc', 'request_id': 'req-6db7dc87-a592-491f-93dc-aa7bc9117682'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.249357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042f54b443c14c41a280db13076b1fdc', 'request_id': 'req-6db7dc87-a592-491f-93dc-aa7bc9117682'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.257896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042f54b443c14c41a280db13076b1fdc', 'request_id': 'req-6db7dc87-a592-491f-93dc-aa7bc9117682'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.263816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 927/1859] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:34 2022] POST /identity/v3/projects => generated 376 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:34.266207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a091f9-6b94-482a-9630-5159dcaebc00 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.272765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] The token cache is being invalidate because project 042f54b443c14c41a280db13076b1fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:34.273271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.273974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.275871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 933/1860] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:34 2022] DELETE /identity/v3/projects/042f54b443c14c41a280db13076b1fdc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:34.279415 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.280789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:34.286912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:34.328921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:34.353216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a091f9-6b94-482a-9630-5159dcaebc00 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.359957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.360763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.361126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7dc87-a592-491f-93dc-aa7bc9117682 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.361777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.362108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.362914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.364818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:34.366679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.367188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.367541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.367889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] PATH_INFO: `/v3/users/8df0bb5f50284c35814513d8cefa002f` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.369162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df0bb5f50284c35814513d8cefa002f)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.371195 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:34.394481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 928/1861] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:34 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:21:34.398637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.400925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:34.420684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cfd4f-4f9f-4b5b-8432-8a7baf48c892 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.428654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8d72-8556-4b9e-b216-05121cb1cfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.430904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8d72-8556-4b9e-b216-05121cb1cfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.431450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8d72-8556-4b9e-b216-05121cb1cfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.432079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8d72-8556-4b9e-b216-05121cb1cfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.437875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8d72-8556-4b9e-b216-05121cb1cfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.440068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8d72-8556-4b9e-b216-05121cb1cfb6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:34.454435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 929/1862] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:34 2022] GET /identity/v3/roles => generated 1427 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:34.458909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8d72-8556-4b9e-b216-05121cb1cfb6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.465770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df0bb5f50284c35814513d8cefa002f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.471997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8d72-8556-4b9e-b216-05121cb1cfb6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.474626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df0bb5f50284c35814513d8cefa002f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.478313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf5c96-713a-4c57-a528-5a8f2994e178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.478547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5c96-713a-4c57-a528-5a8f2994e178 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.478798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5c96-713a-4c57-a528-5a8f2994e178 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.479075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf5c96-713a-4c57-a528-5a8f2994e178 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.479891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf5c96-713a-4c57-a528-5a8f2994e178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.481772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf5c96-713a-4c57-a528-5a8f2994e178 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:34.485905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df0bb5f50284c35814513d8cefa002f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.486903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df0bb5f50284c35814513d8cefa002f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.501175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 930/1863] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:34 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:21:34.501379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf5c96-713a-4c57-a528-5a8f2994e178 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.510265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-39463c1f-e91d-4eb6-bba9-46492e3985c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df0bb5f50284c35814513d8cefa002f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:34.513142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf5c96-713a-4c57-a528-5a8f2994e178 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.522256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 934/1864] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:34 2022] DELETE /identity/v3/users/8df0bb5f50284c35814513d8cefa002f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:34.527115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.528042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.528634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.529083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] PATH_INFO: `/v3/projects/1f4d5f8a33b8419c9e701d04fa1efeac/users/24efac738c784f70b5090f4a3a7acfff/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.551964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4d5f8a33b8419c9e701d04fa1efeac, user_id=24efac738c784f70b5090f4a3a7acfff, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.555490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:34.595262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 931/1865] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:34 2022] PUT /identity/v3/projects/1f4d5f8a33b8419c9e701d04fa1efeac/users/24efac738c784f70b5090f4a3a7acfff/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:34.609733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f3acf-314e-48a9-a571-d9b7b5421b31 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.610170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f3acf-314e-48a9-a571-d9b7b5421b31 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.610632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f3acf-314e-48a9-a571-d9b7b5421b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.669692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6a1f3acf-314e-48a9-a571-d9b7b5421b31 None None] MFA Rules not processed for user `24efac738c784f70b5090f4a3a7acfff`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:34.671157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6a1f3acf-314e-48a9-a571-d9b7b5421b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:34.787248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.789224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 935/1866] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:34.790718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:34.804220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f3acf-314e-48a9-a571-d9b7b5421b31 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:34.809692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6a1f3acf-314e-48a9-a571-d9b7b5421b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:34.877866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:34.976774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f3acf-314e-48a9-a571-d9b7b5421b31 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:34.980879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed513e3-5279-4340-b318-40b544e21f6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:34.981703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed513e3-5279-4340-b318-40b544e21f6b service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:34.982083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed513e3-5279-4340-b318-40b544e21f6b service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:34.982459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed513e3-5279-4340-b318-40b544e21f6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:34.998208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed513e3-5279-4340-b318-40b544e21f6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:34.999714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed513e3-5279-4340-b318-40b544e21f6b service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:35.010748 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feaefd-a281-4ce9-bf26-d02f36a431d7 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:35.014863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 936/1867] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:35.026396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:35.026935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:35.027221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:35.027500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] PATH_INFO: `/v3/users/0822c70d5166473aa74f8522b05fbad3` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:35.033367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0822c70d5166473aa74f8522b05fbad3)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:35.036468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:35.146921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0822c70d5166473aa74f8522b05fbad3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.158236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0822c70d5166473aa74f8522b05fbad3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.159191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0822c70d5166473aa74f8522b05fbad3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.182286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0822c70d5166473aa74f8522b05fbad3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.183179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0822c70d5166473aa74f8522b05fbad3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.217448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 932/1868] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:34 2022] DELETE /identity/v3/users/0822c70d5166473aa74f8522b05fbad3 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:35.221807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a65f7b-9930-47eb-b733-3784b42bb82b None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:35.222354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a65f7b-9930-47eb-b733-3784b42bb82b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:35.222650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a65f7b-9930-47eb-b733-3784b42bb82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:35.260604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e4a65f7b-9930-47eb-b733-3784b42bb82b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:35.262000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e4a65f7b-9930-47eb-b733-3784b42bb82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:35.412488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 937/1869] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:35.422592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:35.425273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:35.470673 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:35.522616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a65f7b-9930-47eb-b733-3784b42bb82b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:35.526751 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e4a65f7b-9930-47eb-b733-3784b42bb82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:35.585329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffb578-e381-4d7d-8203-53a7e73f8c07 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:35.590632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0a4d4-b437-47fc-8535-a6744cfe0f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:35.591358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0a4d4-b437-47fc-8535-a6744cfe0f14 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:35.591588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a65f7b-9930-47eb-b733-3784b42bb82b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:35.591769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0a4d4-b437-47fc-8535-a6744cfe0f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:35.592102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0a4d4-b437-47fc-8535-a6744cfe0f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:35.603676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:35.604385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:35.604635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:35.604924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] PATH_INFO: `/v3/projects/c6ac68040f144d7abcd417d0231370d0` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:35.606351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ac68040f144d7abcd417d0231370d0)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:35.607392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0a4d4-b437-47fc-8535-a6744cfe0f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:35.608434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:35.609484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0a4d4-b437-47fc-8535-a6744cfe0f14 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:35.622282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 933/1870] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:35.683572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ac68040f144d7abcd417d0231370d0', 'request_id': 'req-ce88a614-1747-4ee2-acf9-41ba2010db11'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.701994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ac68040f144d7abcd417d0231370d0', 'request_id': 'req-ce88a614-1747-4ee2-acf9-41ba2010db11'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.717185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ac68040f144d7abcd417d0231370d0', 'request_id': 'req-ce88a614-1747-4ee2-acf9-41ba2010db11'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.745851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] The token cache is being invalidate because project c6ac68040f144d7abcd417d0231370d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:35.746903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.748652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ce88a614-1747-4ee2-acf9-41ba2010db11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:35.754031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 938/1871] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:35 2022] DELETE /identity/v3/projects/c6ac68040f144d7abcd417d0231370d0 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:35.760943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565279b2-6d69-4f8e-9749-8d229e3905e1 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:35.761292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565279b2-6d69-4f8e-9749-8d229e3905e1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:35.761544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565279b2-6d69-4f8e-9749-8d229e3905e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:35.791959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-565279b2-6d69-4f8e-9749-8d229e3905e1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:35.794650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-565279b2-6d69-4f8e-9749-8d229e3905e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:35.898258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 934/1872] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:35.906440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8aa14-59e8-4615-8085-e046cce27ad0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:35.907001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8aa14-59e8-4615-8085-e046cce27ad0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:35.907001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8aa14-59e8-4615-8085-e046cce27ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:35.912354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565279b2-6d69-4f8e-9749-8d229e3905e1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:35.915014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-565279b2-6d69-4f8e-9749-8d229e3905e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:35.932391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-bbe8aa14-59e8-4615-8085-e046cce27ad0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:35.932789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565279b2-6d69-4f8e-9749-8d229e3905e1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:35.933540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bbe8aa14-59e8-4615-8085-e046cce27ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:35.942340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:35.943563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:35.944214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:35.944848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] PATH_INFO: `/v3/users/8cbece7efe6244b9bdfeb28110db5683` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:35.946403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbece7efe6244b9bdfeb28110db5683)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:35.947275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 939/1873] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:35.948186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8aa14-59e8-4615-8085-e046cce27ad0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:35.949259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:35.958734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8aa14-59e8-4615-8085-e046cce27ad0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:35.969166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:35.969997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:35.970514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:35.971769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:35.972727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:35.973140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:35.975053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:35.994585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 940/1874] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:35.999378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.010363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba045f74-3d9f-45fe-ad6a-c89b1b15f846 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:36.020843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:36.021982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.022266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.022540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.023521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:36.024422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbece7efe6244b9bdfeb28110db5683'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.029461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:36.038300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbece7efe6244b9bdfeb28110db5683'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.039213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbece7efe6244b9bdfeb28110db5683'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.055869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:36.065388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbece7efe6244b9bdfeb28110db5683'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.067293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbece7efe6244b9bdfeb28110db5683'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.080630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 941/1875] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:35 2022] POST /identity/v3/projects => generated 382 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:36.081046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.084796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:36.094973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 935/1876] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:35 2022] DELETE /identity/v3/users/8cbece7efe6244b9bdfeb28110db5683 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:36.115692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.117323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:36.162374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:36.226315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2eb906-c472-4bb2-921b-fc3e01d6c142 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:36.233897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69a69-62f8-47ab-a89d-583dd505430e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:36.238122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9eb50-2def-4b3a-ab0c-4fa17bffab66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:36.240439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9eb50-2def-4b3a-ab0c-4fa17bffab66 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.240894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9eb50-2def-4b3a-ab0c-4fa17bffab66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.241364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9eb50-2def-4b3a-ab0c-4fa17bffab66 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.242712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9eb50-2def-4b3a-ab0c-4fa17bffab66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:36.245054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9eb50-2def-4b3a-ab0c-4fa17bffab66 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:36.246393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:36.247222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.247607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.248260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] PATH_INFO: `/v3/projects/733f3be9ee6d45e795c8ef063b58d765` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.249680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=733f3be9ee6d45e795c8ef063b58d765)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:36.252335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:36.277879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 936/1877] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:36 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:36.279769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d9237-15b2-445b-a6be-ac14646ec724 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.280259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d9237-15b2-445b-a6be-ac14646ec724 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.281124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d9237-15b2-445b-a6be-ac14646ec724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.306556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-065d9237-15b2-445b-a6be-ac14646ec724 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:36.308629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-065d9237-15b2-445b-a6be-ac14646ec724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:36.320356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 937/1878] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:36.321571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d9237-15b2-445b-a6be-ac14646ec724 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.357976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d9237-15b2-445b-a6be-ac14646ec724 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:36.358208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733f3be9ee6d45e795c8ef063b58d765', 'request_id': 'req-5b2c8065-0c7e-4643-896d-c8eeb2a03972'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.367853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733f3be9ee6d45e795c8ef063b58d765', 'request_id': 'req-5b2c8065-0c7e-4643-896d-c8eeb2a03972'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.385789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733f3be9ee6d45e795c8ef063b58d765', 'request_id': 'req-5b2c8065-0c7e-4643-896d-c8eeb2a03972'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.403745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] The token cache is being invalidate because project 733f3be9ee6d45e795c8ef063b58d765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:36.404492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.405485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.410517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 942/1879] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:36 2022] DELETE /identity/v3/projects/733f3be9ee6d45e795c8ef063b58d765 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:36.412270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.417013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:36.479196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:36.480511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.480905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.481284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.482598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:36.484688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:36.507416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 938/1880] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:36 2022] GET /identity/v3/roles => generated 1427 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:36.507416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.507924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:36.524192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c06f-2b47-4667-b255-8ea0ac8b80d5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:36.530140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c8065-0c7e-4643-896d-c8eeb2a03972 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:36.530987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:36.531718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.532148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.532686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] PATH_INFO: `/v3/users/caa47ad67bbe42ecaa36e469926feb47` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.535006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa47ad67bbe42ecaa36e469926feb47)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:36.536002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:36.536712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.537369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.537842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:36.538155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.540374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:36.565804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:36.567851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:36.583359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 943/1881] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:36.584726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.587921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:36.604197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc18ba1-a8b7-4834-8654-b46a3457f36e service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:36.615776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2940bd-8d92-42ad-af77-e42c6d3a44c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:36.617260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2940bd-8d92-42ad-af77-e42c6d3a44c5 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.617868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2940bd-8d92-42ad-af77-e42c6d3a44c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.618382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2940bd-8d92-42ad-af77-e42c6d3a44c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.619412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa47ad67bbe42ecaa36e469926feb47'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.620048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2940bd-8d92-42ad-af77-e42c6d3a44c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:36.622227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2940bd-8d92-42ad-af77-e42c6d3a44c5 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:36.633911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa47ad67bbe42ecaa36e469926feb47'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.634574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa47ad67bbe42ecaa36e469926feb47'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.642120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 944/1882] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:36 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:36.650479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2940bd-8d92-42ad-af77-e42c6d3a44c5 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.661952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa47ad67bbe42ecaa36e469926feb47'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.663802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0c673923-ec6d-4382-9358-77176cad2a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa47ad67bbe42ecaa36e469926feb47'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:36.672642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2940bd-8d92-42ad-af77-e42c6d3a44c5 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:36.683874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:36.684820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.685204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.685653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] PATH_INFO: `/v3/projects/f56274e88d574e3fa8d60d4f519e243a/users/7bae870afb964527a932d1fdaf1fb00d/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.709111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56274e88d574e3fa8d60d4f519e243a, user_id=7bae870afb964527a932d1fdaf1fb00d, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:36.711405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 939/1883] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:36 2022] DELETE /identity/v3/users/caa47ad67bbe42ecaa36e469926feb47 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:36.713387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:36.718113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:36.754654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 945/1884] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:36 2022] PUT /identity/v3/projects/f56274e88d574e3fa8d60d4f519e243a/users/7bae870afb964527a932d1fdaf1fb00d/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:36.756151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6ed29-34a8-4570-82e9-b9ed122c74db None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:36.756545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6ed29-34a8-4570-82e9-b9ed122c74db None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:36.756916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6ed29-34a8-4570-82e9-b9ed122c74db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:36.813284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-58e6ed29-34a8-4570-82e9-b9ed122c74db None None] MFA Rules not processed for user `7bae870afb964527a932d1fdaf1fb00d`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:36.816624 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-58e6ed29-34a8-4570-82e9-b9ed122c74db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:36.825633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:36.942463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 940/1885] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:36.947487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:36.949347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:37.082832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5797-c1b5-4b85-b93e-916202c8fb66 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:37.095284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:37.098080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:37.099520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:37.099821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:37.100512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:37.106358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:37.153889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6ed29-34a8-4570-82e9-b9ed122c74db None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:37.161493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 946/1886] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:36 2022] POST /identity/v3/projects => generated 380 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:37.169452 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6ed29-34a8-4570-82e9-b9ed122c74db None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:37.204303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:37.207319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:37.291134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdc794-daa0-4687-8331-343a100c7a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:37.291854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdc794-daa0-4687-8331-343a100c7a32 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:37.292207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdc794-daa0-4687-8331-343a100c7a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:37.292551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdc794-daa0-4687-8331-343a100c7a32 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:37.293867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdc794-daa0-4687-8331-343a100c7a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:37.295915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdc794-daa0-4687-8331-343a100c7a32 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:37.336576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 941/1887] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:37 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:37.341246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdc794-daa0-4687-8331-343a100c7a32 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:37.345866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5715796-f82a-4419-bf8b-42c5f6ebc25c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:37.352725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdc794-daa0-4687-8331-343a100c7a32 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:37.359648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:37.360317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:37.360600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:37.361615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] PATH_INFO: `/v3/projects/f9082cd027184f858a0507fb6688b05d` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:37.362975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9082cd027184f858a0507fb6688b05d)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:37.364510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b52504-9464-4907-81be-72d393df13be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:37.365341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b52504-9464-4907-81be-72d393df13be admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:37.365477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:37.365637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b52504-9464-4907-81be-72d393df13be admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:37.365950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b52504-9464-4907-81be-72d393df13be admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:37.366646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b52504-9464-4907-81be-72d393df13be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:37.368276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b52504-9464-4907-81be-72d393df13be admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:37.391071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 942/1888] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:37 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:37.394780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b52504-9464-4907-81be-72d393df13be admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:37.404982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b52504-9464-4907-81be-72d393df13be admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:37.436888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9082cd027184f858a0507fb6688b05d', 'request_id': 'req-692528d6-4c6f-4c64-833d-90949b777456'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.453270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9082cd027184f858a0507fb6688b05d', 'request_id': 'req-692528d6-4c6f-4c64-833d-90949b777456'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.472727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9082cd027184f858a0507fb6688b05d', 'request_id': 'req-692528d6-4c6f-4c64-833d-90949b777456'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.490168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] The token cache is being invalidate because project f9082cd027184f858a0507fb6688b05d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:37.490447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.491146 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.496164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 947/1889] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:37 2022] DELETE /identity/v3/projects/f9082cd027184f858a0507fb6688b05d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:37.497263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:37.502230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:37.519447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:37.537904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:37.540453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:37.542392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:37.543645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] PATH_INFO: `/v3/projects/8678a207fcf84ef1b139238486d43e46/users/62c504c8840345368d4d089d143aea09/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:37.581930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8678a207fcf84ef1b139238486d43e46, user_id=62c504c8840345368d4d089d143aea09, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:37.584649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:37.615780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 943/1890] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:37 2022] PUT /identity/v3/projects/8678a207fcf84ef1b139238486d43e46/users/62c504c8840345368d4d089d143aea09/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:37.617886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:37.619907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:37.636685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:37.664006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692528d6-4c6f-4c64-833d-90949b777456 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:37.742701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a35d-e464-4e01-b996-0a30e022a41e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:37.747464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:37.748402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:37.748678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:37.749095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] PATH_INFO: `/v3/projects/cbd06b32fbdd43d4b2efd5529f4d2b14` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:37.750301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd06b32fbdd43d4b2efd5529f4d2b14)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:37.752534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:37.754620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:37.755286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:37.755579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:37.755838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] PATH_INFO: `/v3/users/d94574421e004f8ebf9f60749bea3259` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:37.758141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d94574421e004f8ebf9f60749bea3259)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:37.761041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:37.824944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd06b32fbdd43d4b2efd5529f4d2b14', 'request_id': 'req-2ee56851-226b-4841-914d-eddb2a969eb3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.829715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd94574421e004f8ebf9f60749bea3259'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.835178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd06b32fbdd43d4b2efd5529f4d2b14', 'request_id': 'req-2ee56851-226b-4841-914d-eddb2a969eb3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.839576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd94574421e004f8ebf9f60749bea3259'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.846432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd06b32fbdd43d4b2efd5529f4d2b14', 'request_id': 'req-2ee56851-226b-4841-914d-eddb2a969eb3'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.850102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd94574421e004f8ebf9f60749bea3259'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.851263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd94574421e004f8ebf9f60749bea3259'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.862749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] The token cache is being invalidate because project cbd06b32fbdd43d4b2efd5529f4d2b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:37.863048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.864384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.868229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 944/1891] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:37 2022] DELETE /identity/v3/projects/cbd06b32fbdd43d4b2efd5529f4d2b14 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:37.870037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:37.873563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:37.875867 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-624705b4-fa57-4fb7-94b7-3abe5adb9e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd94574421e004f8ebf9f60749bea3259'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:37.884420 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 948/1892] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:37 2022] DELETE /identity/v3/users/d94574421e004f8ebf9f60749bea3259 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:37.886521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9890cb-2d2c-4909-988f-e01fc81436be None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:37.886987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9890cb-2d2c-4909-988f-e01fc81436be None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:37.887510 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9890cb-2d2c-4909-988f-e01fc81436be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:37.915438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7f9890cb-2d2c-4909-988f-e01fc81436be None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:37.916927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7f9890cb-2d2c-4909-988f-e01fc81436be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:37.954604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:37.995442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 949/1893] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:38.010946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc1e7-18a9-4ef5-8298-20560d5e0312 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.011335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc1e7-18a9-4ef5-8298-20560d5e0312 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.011620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc1e7-18a9-4ef5-8298-20560d5e0312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.014916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee56851-226b-4841-914d-eddb2a969eb3 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.021509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.022385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.023018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.023450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.024223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.025990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.040572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a69cc1e7-18a9-4ef5-8298-20560d5e0312 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:38.042557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a69cc1e7-18a9-4ef5-8298-20560d5e0312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.044547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 945/1894] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:37 2022] GET /identity/v3/roles => generated 1427 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:38.049201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.050991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.057006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 950/1895] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:38.062978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cc1e7-18a9-4ef5-8298-20560d5e0312 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.064313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a53a9-4e58-4f3d-8c06-c82a09c49967 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.069440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66852ee2-8b0f-4222-9d9d-d7f1588de035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.070125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852ee2-8b0f-4222-9d9d-d7f1588de035 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.070838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852ee2-8b0f-4222-9d9d-d7f1588de035 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.070838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852ee2-8b0f-4222-9d9d-d7f1588de035 admin admin] PATH_INFO: `/v3/projects/8678a207fcf84ef1b139238486d43e46/users/62c504c8840345368d4d089d143aea09/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.072823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66852ee2-8b0f-4222-9d9d-d7f1588de035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8678a207fcf84ef1b139238486d43e46, user_id=62c504c8840345368d4d089d143aea09, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.075677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66852ee2-8b0f-4222-9d9d-d7f1588de035 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.080234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cc1e7-18a9-4ef5-8298-20560d5e0312 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.087920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 946/1896] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:38 2022] PUT /identity/v3/projects/8678a207fcf84ef1b139238486d43e46/users/62c504c8840345368d4d089d143aea09/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:38.088201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.089019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.089822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.090031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.091329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:38.091881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.095160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.100343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24969c-d942-479e-a10d-e4e90a1af0ec None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.100634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24969c-d942-479e-a10d-e4e90a1af0ec None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.100912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24969c-d942-479e-a10d-e4e90a1af0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.120652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 951/1897] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:21:38.121654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.133053 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-fd24969c-d942-479e-a10d-e4e90a1af0ec None None] MFA Rules not processed for user `62c504c8840345368d4d089d143aea09`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:38.134167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fd1f3-5249-4714-8cb8-fcc5281b20a2 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.134414 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fd24969c-d942-479e-a10d-e4e90a1af0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.149965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:38.213020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab33d-0565-43bc-b816-b2388b21ac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.213688 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab33d-0565-43bc-b816-b2388b21ac60 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.213969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab33d-0565-43bc-b816-b2388b21ac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.214247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab33d-0565-43bc-b816-b2388b21ac60 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.214902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab33d-0565-43bc-b816-b2388b21ac60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.216997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab33d-0565-43bc-b816-b2388b21ac60 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.229563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 947/1898] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:38.240301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24969c-d942-479e-a10d-e4e90a1af0ec None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.242655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fd24969c-d942-479e-a10d-e4e90a1af0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.250074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 952/1899] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/projects => generated 386 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:38.256459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab33d-0565-43bc-b816-b2388b21ac60 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.322297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24969c-d942-479e-a10d-e4e90a1af0ec None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.352278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab33d-0565-43bc-b816-b2388b21ac60 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.357900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.358546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.358834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.359106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.359779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.362039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.380983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679315a-6440-4f6d-b641-bf9d01b6e981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.381678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679315a-6440-4f6d-b641-bf9d01b6e981 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.381961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679315a-6440-4f6d-b641-bf9d01b6e981 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.382236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679315a-6440-4f6d-b641-bf9d01b6e981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.385838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 953/1900] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:21:38.385838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.387110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.400448 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0b1c9-b1b2-4a85-b977-4469319e1891 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.405521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.406162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.406442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.406719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] PATH_INFO: `/v3/projects/f59b2fc0282a4d2facf1143b4ee16c30` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.408846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f59b2fc0282a4d2facf1143b4ee16c30)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.410971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.443082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:38.451829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679315a-6440-4f6d-b641-bf9d01b6e981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.453316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679315a-6440-4f6d-b641-bf9d01b6e981 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.462440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f59b2fc0282a4d2facf1143b4ee16c30', 'request_id': 'req-7c16dd66-42be-432c-8ab9-3c156949470f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.466763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 948/1901] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:38 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:38.467658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679315a-6440-4f6d-b641-bf9d01b6e981 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.468997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f59b2fc0282a4d2facf1143b4ee16c30', 'request_id': 'req-7c16dd66-42be-432c-8ab9-3c156949470f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.478812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f59b2fc0282a4d2facf1143b4ee16c30', 'request_id': 'req-7c16dd66-42be-432c-8ab9-3c156949470f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.481429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679315a-6440-4f6d-b641-bf9d01b6e981 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.492940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:38.493451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] The token cache is being invalidate because project f59b2fc0282a4d2facf1143b4ee16c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:38.493750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.494542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7c16dd66-42be-432c-8ab9-3c156949470f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.500103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 954/1902] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:38 2022] DELETE /identity/v3/projects/f59b2fc0282a4d2facf1143b4ee16c30 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:38.506989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424d080-680f-49f0-8667-e656b4eb2113 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.507279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424d080-680f-49f0-8667-e656b4eb2113 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.507559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424d080-680f-49f0-8667-e656b4eb2113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.530077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2424d080-680f-49f0-8667-e656b4eb2113 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:38.532277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2424d080-680f-49f0-8667-e656b4eb2113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.556041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.556411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.556694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.556961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.557679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.559271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.570668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 949/1903] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:38 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:38.574468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.576738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.593268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51266e60-5b8b-46ac-864e-d533986bbe8d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.597455 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff629c-ab6f-4640-b4be-336794e977cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.598083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff629c-ab6f-4640-b4be-336794e977cc admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.598364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff629c-ab6f-4640-b4be-336794e977cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.598642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff629c-ab6f-4640-b4be-336794e977cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.599337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff629c-ab6f-4640-b4be-336794e977cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.600951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff629c-ab6f-4640-b4be-336794e977cc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.607816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 955/1904] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:38.616570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 950/1905] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:38 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:21:38.617708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424d080-680f-49f0-8667-e656b4eb2113 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.618098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae9f1-a504-45c3-937b-289a1681af01 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.618623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae9f1-a504-45c3-937b-289a1681af01 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.618901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae9f1-a504-45c3-937b-289a1681af01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.627763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424d080-680f-49f0-8667-e656b4eb2113 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.632439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b337994-34b5-4516-9a85-1bca0e608df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.633127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b337994-34b5-4516-9a85-1bca0e608df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.633402 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b337994-34b5-4516-9a85-1bca0e608df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.633884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b337994-34b5-4516-9a85-1bca0e608df1 admin admin] PATH_INFO: `/v3/projects/dff1d0621a374c15ad868769546fc57a/users/3f72e96ab3a94bf7b27e9baa93e36371/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.641469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e73ae9f1-a504-45c3-937b-289a1681af01 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:38.643086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e73ae9f1-a504-45c3-937b-289a1681af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.650923 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b337994-34b5-4516-9a85-1bca0e608df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff1d0621a374c15ad868769546fc57a, user_id=3f72e96ab3a94bf7b27e9baa93e36371, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.655033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 951/1906] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:38.655289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b337994-34b5-4516-9a85-1bca0e608df1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.658301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ae9f1-a504-45c3-937b-289a1681af01 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.668354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ae9f1-a504-45c3-937b-289a1681af01 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.673204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 956/1907] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:38 2022] PUT /identity/v3/projects/dff1d0621a374c15ad868769546fc57a/users/3f72e96ab3a94bf7b27e9baa93e36371/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:38.685429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eaca2-3432-4f8e-82c0-e3746657e84a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.685868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eaca2-3432-4f8e-82c0-e3746657e84a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.686268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eaca2-3432-4f8e-82c0-e3746657e84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.719430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ec6eaca2-3432-4f8e-82c0-e3746657e84a None None] MFA Rules not processed for user `3f72e96ab3a94bf7b27e9baa93e36371`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:38.721846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ec6eaca2-3432-4f8e-82c0-e3746657e84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.741613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.742525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.742869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.743249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.744524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:38.745316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.751749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.764839 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 952/1908] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:38.765903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.767937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.781717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63567c-a22a-4408-bc2a-71723b462938 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.787534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.788318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.788484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.788850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] PATH_INFO: `/v3/users/24efac738c784f70b5090f4a3a7acfff` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.790098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24efac738c784f70b5090f4a3a7acfff)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.792002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.846471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 957/1909] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:38.847963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6eaca2-3432-4f8e-82c0-e3746657e84a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.871207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6eaca2-3432-4f8e-82c0-e3746657e84a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:38.884075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24efac738c784f70b5090f4a3a7acfff'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.886169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:38.886334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:38.886610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:38.887351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:38.888188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:38.890887 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:38.895736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24efac738c784f70b5090f4a3a7acfff'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.896418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24efac738c784f70b5090f4a3a7acfff'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.914596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24efac738c784f70b5090f4a3a7acfff'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.915382 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24efac738c784f70b5090f4a3a7acfff'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:38.918752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:38.926251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:38.928984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 958/1910] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/projects => generated 386 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:38.929922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.931600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:38.940835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 953/1911] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:38 2022] DELETE /identity/v3/users/24efac738c784f70b5090f4a3a7acfff => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:38.940835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:38.942990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:39.004322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7402093-4230-4176-a157-5caa0399ad57 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.023970 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0558b45-c275-49e3-b59d-501c100af1f6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.028838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c504c8840345368d4d089d143aea09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8678a207fcf84ef1b139238486d43e46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.029501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.029788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.030184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.030890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:39.031197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.034202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.046216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 954/1912] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:39.047057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.049961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:39.082096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c3996-da07-4003-a499-2a73a55b6a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.082650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c3996-da07-4003-a499-2a73a55b6a77 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.082936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c3996-da07-4003-a499-2a73a55b6a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.083219 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c3996-da07-4003-a499-2a73a55b6a77 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.083892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c3996-da07-4003-a499-2a73a55b6a77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.085906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c3996-da07-4003-a499-2a73a55b6a77 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.120945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 959/1913] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:38 2022] POST /identity/v3/projects => generated 388 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:39.122573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab60ee0-2cbf-4779-a9a9-cdec2304a90e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.123115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab60ee0-2cbf-4779-a9a9-cdec2304a90e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.123387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab60ee0-2cbf-4779-a9a9-cdec2304a90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.125610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c854bd7-0f25-4410-908e-0dbf2bbd008b tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.144893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2ab60ee0-2cbf-4779-a9a9-cdec2304a90e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:39.146149 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2ab60ee0-2cbf-4779-a9a9-cdec2304a90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:39.201848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02907d0d-a185-41d9-af2e-a0e2f3f07277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.202480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02907d0d-a185-41d9-af2e-a0e2f3f07277 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.202765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02907d0d-a185-41d9-af2e-a0e2f3f07277 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.203032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02907d0d-a185-41d9-af2e-a0e2f3f07277 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.203700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02907d0d-a185-41d9-af2e-a0e2f3f07277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.206355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02907d0d-a185-41d9-af2e-a0e2f3f07277 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.218501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 960/1914] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:39.219555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab60ee0-2cbf-4779-a9a9-cdec2304a90e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.230654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 955/1915] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:39 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:39.231385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02907d0d-a185-41d9-af2e-a0e2f3f07277 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.241886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02907d0d-a185-41d9-af2e-a0e2f3f07277 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.246705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.247309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.247967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.248245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.248906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.250877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.258281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:39.273325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 956/1916] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:39 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:39.274179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.275796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:39.290327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:39.291349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab60ee0-2cbf-4779-a9a9-cdec2304a90e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.296318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58006083-55b9-4289-b487-1d8af9fa9f40 tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c504c8840345368d4d089d143aea09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8678a207fcf84ef1b139238486d43e46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.297026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58006083-55b9-4289-b487-1d8af9fa9f40 tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.297343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58006083-55b9-4289-b487-1d8af9fa9f40 tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.297680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58006083-55b9-4289-b487-1d8af9fa9f40 tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.298945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58006083-55b9-4289-b487-1d8af9fa9f40 tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.301022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58006083-55b9-4289-b487-1d8af9fa9f40 tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.332162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 961/1917] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:39 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:39.332162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58006083-55b9-4289-b487-1d8af9fa9f40 tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.337688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76716352-4437-4c28-9d91-c73f6b2bfd2d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.387896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6117ee-2ea3-40df-a203-b7abe5b72904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.388525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6117ee-2ea3-40df-a203-b7abe5b72904 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.388860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6117ee-2ea3-40df-a203-b7abe5b72904 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.389142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6117ee-2ea3-40df-a203-b7abe5b72904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.395291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58006083-55b9-4289-b487-1d8af9fa9f40 tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.399439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6117ee-2ea3-40df-a203-b7abe5b72904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.399968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b8302-dce6-4cf2-89ae-ed05a0451ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.400591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b8302-dce6-4cf2-89ae-ed05a0451ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.400868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b8302-dce6-4cf2-89ae-ed05a0451ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.400982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6117ee-2ea3-40df-a203-b7abe5b72904 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.401139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b8302-dce6-4cf2-89ae-ed05a0451ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.401863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b8302-dce6-4cf2-89ae-ed05a0451ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.403570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b8302-dce6-4cf2-89ae-ed05a0451ae6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.410648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 957/1918] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:39.411461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6117ee-2ea3-40df-a203-b7abe5b72904 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.418144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 962/1919] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:39 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:39.419144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b8302-dce6-4cf2-89ae-ed05a0451ae6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.423366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6117ee-2ea3-40df-a203-b7abe5b72904 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.429423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96d113-2f24-42ba-b9ca-2df78b43bcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.430506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d113-2f24-42ba-b9ca-2df78b43bcaf admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.430784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d113-2f24-42ba-b9ca-2df78b43bcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.431058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d113-2f24-42ba-b9ca-2df78b43bcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.431759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96d113-2f24-42ba-b9ca-2df78b43bcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.433357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96d113-2f24-42ba-b9ca-2df78b43bcaf admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.444370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 958/1920] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:39 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:39.445456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96d113-2f24-42ba-b9ca-2df78b43bcaf admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.455573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96d113-2f24-42ba-b9ca-2df78b43bcaf admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.460512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046e48-690b-498a-b120-58a50e5b02c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.461132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046e48-690b-498a-b120-58a50e5b02c8 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.461434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046e48-690b-498a-b120-58a50e5b02c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.461719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046e48-690b-498a-b120-58a50e5b02c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.463049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a046e48-690b-498a-b120-58a50e5b02c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.464667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a046e48-690b-498a-b120-58a50e5b02c8 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.476234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 959/1921] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:39 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:39.481907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046e48-690b-498a-b120-58a50e5b02c8 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.489321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046e48-690b-498a-b120-58a50e5b02c8 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.496225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfea97-34f9-4292-8d45-85ba086577c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.496879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfea97-34f9-4292-8d45-85ba086577c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.497166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfea97-34f9-4292-8d45-85ba086577c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.497503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfea97-34f9-4292-8d45-85ba086577c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.498464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfea97-34f9-4292-8d45-85ba086577c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.500162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfea97-34f9-4292-8d45-85ba086577c4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.500657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b8302-dce6-4cf2-89ae-ed05a0451ae6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.510763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c504c8840345368d4d089d143aea09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8678a207fcf84ef1b139238486d43e46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.511599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.512046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.512457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.513562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 960/1922] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:39 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:39.514106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfea97-34f9-4292-8d45-85ba086577c4 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.518267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:39.518912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.520596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.525107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfea97-34f9-4292-8d45-85ba086577c4 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.530365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e8a4d-69de-4aca-95f0-acdb0e47b45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.531439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e8a4d-69de-4aca-95f0-acdb0e47b45c admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.531439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e8a4d-69de-4aca-95f0-acdb0e47b45c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.531697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e8a4d-69de-4aca-95f0-acdb0e47b45c admin admin] PATH_INFO: `/v3/projects/43d1fd029f3a4963824e93f57cb97809/users/8c22e86703a14a3b986f42f5bd870808/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.539303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 963/1923] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:39.539303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.549163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e8a4d-69de-4aca-95f0-acdb0e47b45c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d1fd029f3a4963824e93f57cb97809, user_id=8c22e86703a14a3b986f42f5bd870808, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.551972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e8a4d-69de-4aca-95f0-acdb0e47b45c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.565880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871695-6244-48f7-acc9-2deb320ee65c tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.571051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 961/1924] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:39 2022] PUT /identity/v3/projects/43d1fd029f3a4963824e93f57cb97809/users/8c22e86703a14a3b986f42f5bd870808/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:39.584686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b013-1d1b-4bde-aade-fa359620270f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.585384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b013-1d1b-4bde-aade-fa359620270f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.591345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1b013-1d1b-4bde-aade-fa359620270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.627572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-26b1b013-1d1b-4bde-aade-fa359620270f None None] MFA Rules not processed for user `8c22e86703a14a3b986f42f5bd870808`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:39.628996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-26b1b013-1d1b-4bde-aade-fa359620270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:39.631387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:39.647921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f08c0-0dbe-43c3-a9d5-436aae59a7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.648597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f08c0-0dbe-43c3-a9d5-436aae59a7c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.648882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f08c0-0dbe-43c3-a9d5-436aae59a7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.649351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f08c0-0dbe-43c3-a9d5-436aae59a7c4 admin admin] PATH_INFO: `/v3/projects/57d3bc56165b41b3afd78f6f979b03c6/users/bab0f2cb28324dc5b9915f89d1f3748a/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.668114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f08c0-0dbe-43c3-a9d5-436aae59a7c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d3bc56165b41b3afd78f6f979b03c6, user_id=bab0f2cb28324dc5b9915f89d1f3748a, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.670905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f08c0-0dbe-43c3-a9d5-436aae59a7c4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.687339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 964/1925] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:39 2022] PUT /identity/v3/projects/57d3bc56165b41b3afd78f6f979b03c6/users/bab0f2cb28324dc5b9915f89d1f3748a/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:39.688627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f08c0-0dbe-43c3-a9d5-436aae59a7c4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.714219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 962/1926] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:39.715571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715f42-4bdf-4f97-9318-fbe6cafa2e70 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.715856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715f42-4bdf-4f97-9318-fbe6cafa2e70 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.716138 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715f42-4bdf-4f97-9318-fbe6cafa2e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.726026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:39.752205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-68715f42-4bdf-4f97-9318-fbe6cafa2e70 None None] MFA Rules not processed for user `bab0f2cb28324dc5b9915f89d1f3748a`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:39.753476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-68715f42-4bdf-4f97-9318-fbe6cafa2e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:39.773150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f08c0-0dbe-43c3-a9d5-436aae59a7c4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.778672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c504c8840345368d4d089d143aea09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8678a207fcf84ef1b139238486d43e46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.779582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.779878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.780158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] PATH_INFO: `/v3/projects/a5a7340809b64055b58940e902b6b76c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.781776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a5a7340809b64055b58940e902b6b76c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.784094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.828019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 963/1927] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:39.828876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68715f42-4bdf-4f97-9318-fbe6cafa2e70 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.831029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-68715f42-4bdf-4f97-9318-fbe6cafa2e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:39.835783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a7340809b64055b58940e902b6b76c', 'request_id': 'req-5317e556-37c0-46ed-ae83-c7bdd4ef984d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:39.841823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a7340809b64055b58940e902b6b76c', 'request_id': 'req-5317e556-37c0-46ed-ae83-c7bdd4ef984d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:39.851025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a7340809b64055b58940e902b6b76c', 'request_id': 'req-5317e556-37c0-46ed-ae83-c7bdd4ef984d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:39.863244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] The token cache is being invalidate because project a5a7340809b64055b58940e902b6b76c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:39.863532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:39.864207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:39.865815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 965/1928] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:39 2022] DELETE /identity/v3/projects/a5a7340809b64055b58940e902b6b76c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:39.866662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:39.867956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:39.904702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68715f42-4bdf-4f97-9318-fbe6cafa2e70 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.947981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317e556-37c0-46ed-ae83-c7bdd4ef984d tempest-VolumeQuotaClassesTest-179213671 tempest-VolumeQuotaClassesTest-179213671-project-admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:39.953266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.954399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.954764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.955251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] PATH_INFO: `/v3/projects/1f4d5f8a33b8419c9e701d04fa1efeac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.956892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4d5f8a33b8419c9e701d04fa1efeac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:39.959327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:39.961059 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:39.963166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:39.964132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:39.964521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:39.964840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:39.967124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.041498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4d5f8a33b8419c9e701d04fa1efeac', 'request_id': 'req-6df75439-951b-43b8-a808-d9a87b59e708'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.050031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4d5f8a33b8419c9e701d04fa1efeac', 'request_id': 'req-6df75439-951b-43b8-a808-d9a87b59e708'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.050507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.052046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.060986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4d5f8a33b8419c9e701d04fa1efeac', 'request_id': 'req-6df75439-951b-43b8-a808-d9a87b59e708'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.065856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 964/1929] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:39 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:40.068910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.070545 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.075985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] The token cache is being invalidate because project 1f4d5f8a33b8419c9e701d04fa1efeac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:40.076311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.077246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6df75439-951b-43b8-a808-d9a87b59e708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.079186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 966/1930] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:39 2022] DELETE /identity/v3/projects/1f4d5f8a33b8419c9e701d04fa1efeac => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:40.082799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:40.084124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfa46-f2d3-4abf-b148-c6de3b841097 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.084494 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfa46-f2d3-4abf-b148-c6de3b841097 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.084771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfa46-f2d3-4abf-b148-c6de3b841097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.089670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b17c6-3589-4d78-a83b-13c39e5be028 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.125861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-cfdbfa46-f2d3-4abf-b148-c6de3b841097 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:40.127262 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cfdbfa46-f2d3-4abf-b148-c6de3b841097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.221865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbe277-734b-4d66-897e-42eb07a8db51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.223299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbe277-734b-4d66-897e-42eb07a8db51 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.223598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbe277-734b-4d66-897e-42eb07a8db51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.224006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbe277-734b-4d66-897e-42eb07a8db51 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.224815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbe277-734b-4d66-897e-42eb07a8db51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.226976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbe277-734b-4d66-897e-42eb07a8db51 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.252161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 967/1931] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:40.253482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbfa46-f2d3-4abf-b148-c6de3b841097 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.257105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cfdbfa46-f2d3-4abf-b148-c6de3b841097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.271029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 965/1932] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:40 2022] POST /identity/v3/projects => generated 388 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:40.271029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256ee84-ab6d-4c27-955c-37c1c7a59df1 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.271029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256ee84-ab6d-4c27-955c-37c1c7a59df1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.271029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256ee84-ab6d-4c27-955c-37c1c7a59df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.280195 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbfa46-f2d3-4abf-b148-c6de3b841097 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.292082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b256ee84-ab6d-4c27-955c-37c1c7a59df1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:40.293455 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b256ee84-ab6d-4c27-955c-37c1c7a59df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.360391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.361939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.362349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.362704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] PATH_INFO: `/v3/users/7bae870afb964527a932d1fdaf1fb00d` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.364270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bae870afb964527a932d1fdaf1fb00d)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.367302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.380578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 966/1933] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:40.381871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256ee84-ab6d-4c27-955c-37c1c7a59df1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.383219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b256ee84-ab6d-4c27-955c-37c1c7a59df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.425943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256ee84-ab6d-4c27-955c-37c1c7a59df1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.440925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62d045-13ef-4ccb-9500-e3637aba2527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.441597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:40.442606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62d045-13ef-4ccb-9500-e3637aba2527 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.442886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62d045-13ef-4ccb-9500-e3637aba2527 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.443322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62d045-13ef-4ccb-9500-e3637aba2527 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.444441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62d045-13ef-4ccb-9500-e3637aba2527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.447362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62d045-13ef-4ccb-9500-e3637aba2527 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.461090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bae870afb964527a932d1fdaf1fb00d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.480104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bae870afb964527a932d1fdaf1fb00d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.492364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bae870afb964527a932d1fdaf1fb00d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.493124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bae870afb964527a932d1fdaf1fb00d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.494777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 967/1934] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:40 2022] POST /identity/v3/users => generated 301 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:40.495802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62d045-13ef-4ccb-9500-e3637aba2527 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.510810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62d045-13ef-4ccb-9500-e3637aba2527 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.512604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bae870afb964527a932d1fdaf1fb00d'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:40.516103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30e3c3-e4de-431d-bef6-1304f57db877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.517104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e3c3-e4de-431d-bef6-1304f57db877 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.521963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 968/1935] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:40 2022] DELETE /identity/v3/users/7bae870afb964527a932d1fdaf1fb00d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:40.524695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.524695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.531524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e3c3-e4de-431d-bef6-1304f57db877 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.531807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e3c3-e4de-431d-bef6-1304f57db877 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.532789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30e3c3-e4de-431d-bef6-1304f57db877 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:40.533746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30e3c3-e4de-431d-bef6-1304f57db877 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.535458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30e3c3-e4de-431d-bef6-1304f57db877 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.551038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 968/1936] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:40.552153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf31ebb-51ed-4c34-aabb-bce2b9386c9b None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.552441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf31ebb-51ed-4c34-aabb-bce2b9386c9b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.552721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf31ebb-51ed-4c34-aabb-bce2b9386c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.573346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-fcf31ebb-51ed-4c34-aabb-bce2b9386c9b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:40.574650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fcf31ebb-51ed-4c34-aabb-bce2b9386c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.612898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c65853-cfdc-4b90-86f4-ad986d4b9e4c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.617918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.618547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.618823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.619089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.619786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.621395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.634391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 969/1937] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:40 2022] GET /identity/v3/roles => generated 1427 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:40.635426 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.637222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.651294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:40.652019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c2c0f-d43d-44c8-9fe7-f49c7e3d3250 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.661230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.661937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.662182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.664223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.664922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.665568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 969/1938] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:40.666845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf31ebb-51ed-4c34-aabb-bce2b9386c9b None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.670671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.685258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf31ebb-51ed-4c34-aabb-bce2b9386c9b None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.691192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25014c54-1e42-490b-8b9b-281f346d586c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.691947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25014c54-1e42-490b-8b9b-281f346d586c admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.692377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25014c54-1e42-490b-8b9b-281f346d586c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.692666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25014c54-1e42-490b-8b9b-281f346d586c admin admin] PATH_INFO: `/v3/projects/d113d20821bd4caca9802e0b3e6a08b8/users/b3116a6f63d84f6e91e9f9c69468ae90/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.705893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 970/1939] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:40 2022] POST /identity/v3/projects => generated 374 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:40.706819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.710646 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:40.711779 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25014c54-1e42-490b-8b9b-281f346d586c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d113d20821bd4caca9802e0b3e6a08b8, user_id=b3116a6f63d84f6e91e9f9c69468ae90, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.714532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25014c54-1e42-490b-8b9b-281f346d586c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.733092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 970/1940] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:40 2022] PUT /identity/v3/projects/d113d20821bd4caca9802e0b3e6a08b8/users/b3116a6f63d84f6e91e9f9c69468ae90/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:40.733092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25014c54-1e42-490b-8b9b-281f346d586c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.795069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71896cc1-c01e-4e2c-947c-478d44974e7b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.805030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25014c54-1e42-490b-8b9b-281f346d586c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.810170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cbaa3-b9dc-4e12-b366-f4dd52a6f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.810803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cbaa3-b9dc-4e12-b366-f4dd52a6f17e admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.811084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cbaa3-b9dc-4e12-b366-f4dd52a6f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.811359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cbaa3-b9dc-4e12-b366-f4dd52a6f17e admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.812031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cbaa3-b9dc-4e12-b366-f4dd52a6f17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.814280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cbaa3-b9dc-4e12-b366-f4dd52a6f17e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.844788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 971/1941] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:40 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:40.844788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cbaa3-b9dc-4e12-b366-f4dd52a6f17e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.855440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cbaa3-b9dc-4e12-b366-f4dd52a6f17e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.859312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:40.865185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623efe18-b6ec-48d2-abaf-e544e3099a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.865884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623efe18-b6ec-48d2-abaf-e544e3099a96 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.866172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623efe18-b6ec-48d2-abaf-e544e3099a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.866601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623efe18-b6ec-48d2-abaf-e544e3099a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.867315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623efe18-b6ec-48d2-abaf-e544e3099a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.870943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623efe18-b6ec-48d2-abaf-e544e3099a96 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.885952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e50123-069d-453c-97ae-c6f9a207f82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.889710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e50123-069d-453c-97ae-c6f9a207f82c service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.889857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 972/1942] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:40 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:40.889857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623efe18-b6ec-48d2-abaf-e544e3099a96 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.890044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e50123-069d-453c-97ae-c6f9a207f82c service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.890044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e50123-069d-453c-97ae-c6f9a207f82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.902897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623efe18-b6ec-48d2-abaf-e544e3099a96 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.903135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e50123-069d-453c-97ae-c6f9a207f82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.904830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e50123-069d-453c-97ae-c6f9a207f82c service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.907684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4de94-96fe-43b4-8236-30f0445f13d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.908818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4de94-96fe-43b4-8236-30f0445f13d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.908818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4de94-96fe-43b4-8236-30f0445f13d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.908818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4de94-96fe-43b4-8236-30f0445f13d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.909672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4de94-96fe-43b4-8236-30f0445f13d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.912401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4de94-96fe-43b4-8236-30f0445f13d0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.917793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 971/1943] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:40.918974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e50123-069d-453c-97ae-c6f9a207f82c service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.926877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 973/1944] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:40 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:40.933014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e50123-069d-453c-97ae-c6f9a207f82c service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.933270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4de94-96fe-43b4-8236-30f0445f13d0 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:40.939735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fb69f-8c6e-44af-8083-f3c108d95a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:40.940430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fb69f-8c6e-44af-8083-f3c108d95a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.940712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fb69f-8c6e-44af-8083-f3c108d95a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.941008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fb69f-8c6e-44af-8083-f3c108d95a2b admin admin] PATH_INFO: `/v3/projects/d113d20821bd4caca9802e0b3e6a08b8/users/b3116a6f63d84f6e91e9f9c69468ae90/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:40.943182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fb69f-8c6e-44af-8083-f3c108d95a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d113d20821bd4caca9802e0b3e6a08b8, user_id=b3116a6f63d84f6e91e9f9c69468ae90, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:40.946485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fb69f-8c6e-44af-8083-f3c108d95a2b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:40.958219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4de94-96fe-43b4-8236-30f0445f13d0 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:40.971324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 972/1945] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:40 2022] PUT /identity/v3/projects/d113d20821bd4caca9802e0b3e6a08b8/users/b3116a6f63d84f6e91e9f9c69468ae90/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:40.981193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e97c-890e-4ab5-b149-e86997975710 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:40.981518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e97c-890e-4ab5-b149-e86997975710 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:40.981806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e97c-890e-4ab5-b149-e86997975710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:41.024311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-06f5e97c-890e-4ab5-b149-e86997975710 None None] MFA Rules not processed for user `b3116a6f63d84f6e91e9f9c69468ae90`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:41.025708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-06f5e97c-890e-4ab5-b149-e86997975710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:41.046740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f282fc-832b-4c56-ba5a-0b2ec02852d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:41.047379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f282fc-832b-4c56-ba5a-0b2ec02852d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:41.047751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f282fc-832b-4c56-ba5a-0b2ec02852d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:41.048083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f282fc-832b-4c56-ba5a-0b2ec02852d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:41.048894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f282fc-832b-4c56-ba5a-0b2ec02852d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:41.050719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f282fc-832b-4c56-ba5a-0b2ec02852d4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:41.062245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 974/1946] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:40 2022] GET /identity/v3/roles => generated 1427 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:41.066886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f282fc-832b-4c56-ba5a-0b2ec02852d4 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:41.076679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f282fc-832b-4c56-ba5a-0b2ec02852d4 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:41.082549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5751e5-54b5-4911-9020-e99e963c7545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:41.083413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5751e5-54b5-4911-9020-e99e963c7545 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:41.083653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5751e5-54b5-4911-9020-e99e963c7545 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:41.083938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5751e5-54b5-4911-9020-e99e963c7545 admin admin] PATH_INFO: `/v3/projects/9017e927b016427aa84b863e0e2a86a4/users/9542521c42b34769a236eb6f268c5358/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:41.108161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5751e5-54b5-4911-9020-e99e963c7545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9017e927b016427aa84b863e0e2a86a4, user_id=9542521c42b34769a236eb6f268c5358, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:41.117138 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 973/1947] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:40 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:41.119546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5751e5-54b5-4911-9020-e99e963c7545 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:41.128212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5e97c-890e-4ab5-b149-e86997975710 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:41.130055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-06f5e97c-890e-4ab5-b149-e86997975710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:41.146121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 975/1948] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:41 2022] PUT /identity/v3/projects/9017e927b016427aa84b863e0e2a86a4/users/9542521c42b34769a236eb6f268c5358/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:41.151147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b662-0c8e-4b47-94e6-1c6db3afddbc None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:41.151672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b662-0c8e-4b47-94e6-1c6db3afddbc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:41.152166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b662-0c8e-4b47-94e6-1c6db3afddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:41.176771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-4035b662-0c8e-4b47-94e6-1c6db3afddbc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:41.178075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4035b662-0c8e-4b47-94e6-1c6db3afddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:41.192752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:41.208409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5e97c-890e-4ab5-b149-e86997975710 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:41.273750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 976/1949] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:41.274822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035b662-0c8e-4b47-94e6-1c6db3afddbc None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:41.277095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4035b662-0c8e-4b47-94e6-1c6db3afddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:41.289336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:41.290426 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:41.290786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:41.291170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:41.293207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035b662-0c8e-4b47-94e6-1c6db3afddbc None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:41.299747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:41.300423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:41.300652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:41.300911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] PATH_INFO: `/v3/projects/f56274e88d574e3fa8d60d4f519e243a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:41.302113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f56274e88d574e3fa8d60d4f519e243a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:41.304199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:41.324725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:41.371374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:41.374290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:41.375391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f56274e88d574e3fa8d60d4f519e243a', 'request_id': 'req-b039a96f-2477-4fd0-8772-212a24e21feb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.385514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f56274e88d574e3fa8d60d4f519e243a', 'request_id': 'req-b039a96f-2477-4fd0-8772-212a24e21feb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.386830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 974/1950] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:41 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:41.387758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:41.389200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:41.393030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f56274e88d574e3fa8d60d4f519e243a', 'request_id': 'req-b039a96f-2477-4fd0-8772-212a24e21feb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.408578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] The token cache is being invalidate because project f56274e88d574e3fa8d60d4f519e243a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:41.408874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.409610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.412683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 977/1951] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:41 2022] DELETE /identity/v3/projects/f56274e88d574e3fa8d60d4f519e243a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:41.416271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:41.417800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:41.461058 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9b395-9c00-4708-8e09-c881f1303b30 service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:41.524978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a96f-2477-4fd0-8772-212a24e21feb admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:41.532268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:41.532985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:41.533277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:41.533627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:41.533910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] PATH_INFO: `/v3/users/62c504c8840345368d4d089d143aea09` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:41.534012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:41.534643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:41.535145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:41.536065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c504c8840345368d4d089d143aea09)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:41.536769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:41.538178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:41.550252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:41.551613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:41.569390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 975/1952] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:41.597686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:41.604245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c504c8840345368d4d089d143aea09'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.616808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c504c8840345368d4d089d143aea09'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.618286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c504c8840345368d4d089d143aea09'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.645362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c504c8840345368d4d089d143aea09'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.646787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c504c8840345368d4d089d143aea09'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:41.677486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 978/1953] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:41 2022] DELETE /identity/v3/users/62c504c8840345368d4d089d143aea09 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:42.250579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:42.269920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdfad4-4c6b-4752-ac08-2d8272799f23 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:42.271586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:42.273610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:42.367488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:42.368332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:42.368616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:42.368884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] PATH_INFO: `/v3/projects/8678a207fcf84ef1b139238486d43e46` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:42.370209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8678a207fcf84ef1b139238486d43e46)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:42.372208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:42.388684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb3bc9-3e02-4dbb-963a-cb127c3a767d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:42.395259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25589f3f-d8aa-4508-ac86-43410e1da1ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:42.396332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589f3f-d8aa-4508-ac86-43410e1da1ff service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:42.396626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589f3f-d8aa-4508-ac86-43410e1da1ff service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:42.396908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589f3f-d8aa-4508-ac86-43410e1da1ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:42.400362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:42.400987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-25589f3f-d8aa-4508-ac86-43410e1da1ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:42.442054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8678a207fcf84ef1b139238486d43e46', 'request_id': 'req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:42.450595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8678a207fcf84ef1b139238486d43e46', 'request_id': 'req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:42.460768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8678a207fcf84ef1b139238486d43e46', 'request_id': 'req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:42.474839 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25589f3f-d8aa-4508-ac86-43410e1da1ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:42.476492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25589f3f-d8aa-4508-ac86-43410e1da1ff service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:42.480638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] The token cache is being invalidate because project 8678a207fcf84ef1b139238486d43e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:42.481572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:42.482375 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3fa8b0d5-6776-4f47-b6d5-d95a8c9e8e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:42.487179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 976/1954] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:42 2022] DELETE /identity/v3/projects/8678a207fcf84ef1b139238486d43e46 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:42.488407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 979/1955] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:42 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:42.492133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44ee05-2761-4b0c-8b0f-8fce5e6b6766 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:42.492645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44ee05-2761-4b0c-8b0f-8fce5e6b6766 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:42.493117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44ee05-2761-4b0c-8b0f-8fce5e6b6766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:42.515427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-de44ee05-2761-4b0c-8b0f-8fce5e6b6766 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:42.516695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-de44ee05-2761-4b0c-8b0f-8fce5e6b6766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:42.610189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 977/1956] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:42.611446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee7408-0763-4cc9-8049-cebbe2e27739 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:42.611981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee7408-0763-4cc9-8049-cebbe2e27739 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:42.612103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee7408-0763-4cc9-8049-cebbe2e27739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:42.641196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d58b-f2c3-44c3-a140-fdcd9a9ef456 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:42.641512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d58b-f2c3-44c3-a140-fdcd9a9ef456 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:42.642410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d58b-f2c3-44c3-a140-fdcd9a9ef456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:42.647279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-14ee7408-0763-4cc9-8049-cebbe2e27739 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:42.650770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-14ee7408-0763-4cc9-8049-cebbe2e27739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:42.664012 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 980/1957] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:42.670853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee7408-0763-4cc9-8049-cebbe2e27739 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:42.684079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee7408-0763-4cc9-8049-cebbe2e27739 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:42.690662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-c182d58b-f2c3-44c3-a140-fdcd9a9ef456 None None] MFA Rules not processed for user `9542521c42b34769a236eb6f268c5358`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:42.691873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:42.692519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:42.692793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:42.693025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c182d58b-f2c3-44c3-a140-fdcd9a9ef456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:42.693260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:42.694410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:42.694719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:42.696670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:42.719843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 981/1958] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:42.721672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:42.738762 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777d146-2a8b-42d6-a459-0f41be7e84f3 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:42.749256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be91580-6f64-4f59-80cb-2e25996d6e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:42.749955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be91580-6f64-4f59-80cb-2e25996d6e3f admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:42.750265 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be91580-6f64-4f59-80cb-2e25996d6e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:42.750577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be91580-6f64-4f59-80cb-2e25996d6e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:42.751324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be91580-6f64-4f59-80cb-2e25996d6e3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:42.754283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be91580-6f64-4f59-80cb-2e25996d6e3f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:42.805985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 982/1959] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:42 2022] POST /identity/v3/projects => generated 366 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:42.811584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be91580-6f64-4f59-80cb-2e25996d6e3f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:42.836780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 978/1960] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:42.849001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182d58b-f2c3-44c3-a140-fdcd9a9ef456 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:42.851650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c182d58b-f2c3-44c3-a140-fdcd9a9ef456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:42.928063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:42.945826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:42.984363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be91580-6f64-4f59-80cb-2e25996d6e3f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:42.986342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182d58b-f2c3-44c3-a140-fdcd9a9ef456 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.018381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ebc8e-5cdc-4539-8056-cfc1a2ae2cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.019027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ebc8e-5cdc-4539-8056-cfc1a2ae2cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.019302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ebc8e-5cdc-4539-8056-cfc1a2ae2cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.019568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ebc8e-5cdc-4539-8056-cfc1a2ae2cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.020245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ebc8e-5cdc-4539-8056-cfc1a2ae2cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.022237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ebc8e-5cdc-4539-8056-cfc1a2ae2cd1 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.048936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 983/1961] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:42 2022] POST /identity/v3/users => generated 284 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:43.049368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6132e6-426a-49b1-877e-51afba3d08d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.050224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ebc8e-5cdc-4539-8056-cfc1a2ae2cd1 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.050647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6132e6-426a-49b1-877e-51afba3d08d6 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.050995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6132e6-426a-49b1-877e-51afba3d08d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.051189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6132e6-426a-49b1-877e-51afba3d08d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.061321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ebc8e-5cdc-4539-8056-cfc1a2ae2cd1 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.068828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fef68-2993-4d40-8866-0ceb1ffde693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.069934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fef68-2993-4d40-8866-0ceb1ffde693 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.070445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fef68-2993-4d40-8866-0ceb1ffde693 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.070955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fef68-2993-4d40-8866-0ceb1ffde693 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.072267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fef68-2993-4d40-8866-0ceb1ffde693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.075501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fef68-2993-4d40-8866-0ceb1ffde693 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.091610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 984/1962] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:43 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:43.092480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fef68-2993-4d40-8866-0ceb1ffde693 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.113212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fef68-2993-4d40-8866-0ceb1ffde693 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.118352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c00f2-1438-4899-954c-d98848c124da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.118911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c00f2-1438-4899-954c-d98848c124da admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.119187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c00f2-1438-4899-954c-d98848c124da admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.119521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c00f2-1438-4899-954c-d98848c124da admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.119626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6132e6-426a-49b1-877e-51afba3d08d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.120213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c00f2-1438-4899-954c-d98848c124da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.121015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6132e6-426a-49b1-877e-51afba3d08d6 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.121836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c00f2-1438-4899-954c-d98848c124da admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.131028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 979/1963] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:43.137522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 985/1964] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:43 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:43.138363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6132e6-426a-49b1-877e-51afba3d08d6 service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.149380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6132e6-426a-49b1-877e-51afba3d08d6 service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.154104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61012b08-5b53-497a-a751-d0137aebda89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.154858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012b08-5b53-497a-a751-d0137aebda89 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.155144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012b08-5b53-497a-a751-d0137aebda89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.155415 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012b08-5b53-497a-a751-d0137aebda89 admin admin] PATH_INFO: `/v3/projects/41a48a8a5330451788a9de67e1477f13/users/7c75350e018a4b2f8b481be85dc88d9e/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.171218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61012b08-5b53-497a-a751-d0137aebda89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a48a8a5330451788a9de67e1477f13, user_id=7c75350e018a4b2f8b481be85dc88d9e, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.174542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61012b08-5b53-497a-a751-d0137aebda89 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.187163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 980/1965] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:43 2022] PUT /identity/v3/projects/41a48a8a5330451788a9de67e1477f13/users/7c75350e018a4b2f8b481be85dc88d9e/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:43.198530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb4f53-ecfa-4f0e-b034-b5c761329df4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.198814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb4f53-ecfa-4f0e-b034-b5c761329df4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.199089 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb4f53-ecfa-4f0e-b034-b5c761329df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.227007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8ffb4f53-ecfa-4f0e-b034-b5c761329df4 None None] MFA Rules not processed for user `7c75350e018a4b2f8b481be85dc88d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:43.228254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8ffb4f53-ecfa-4f0e-b034-b5c761329df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:43.300086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 986/1966] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:43.304376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61012b08-5b53-497a-a751-d0137aebda89 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.381035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61012b08-5b53-497a-a751-d0137aebda89 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.402147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e8c5-9aaf-49af-a19e-83fa0a915d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.402780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e8c5-9aaf-49af-a19e-83fa0a915d63 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.403112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e8c5-9aaf-49af-a19e-83fa0a915d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.403388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e8c5-9aaf-49af-a19e-83fa0a915d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.404091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5e8c5-9aaf-49af-a19e-83fa0a915d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.406078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5e8c5-9aaf-49af-a19e-83fa0a915d63 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.437570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 981/1967] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:43 2022] POST /identity/v3/projects => generated 364 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:43.439655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb4f53-ecfa-4f0e-b034-b5c761329df4 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.456359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb4f53-ecfa-4f0e-b034-b5c761329df4 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.518029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbead26-beed-4aa8-8b56-2b4d03d7b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.518668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbead26-beed-4aa8-8b56-2b4d03d7b657 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.518968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbead26-beed-4aa8-8b56-2b4d03d7b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.519244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbead26-beed-4aa8-8b56-2b4d03d7b657 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.519906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbead26-beed-4aa8-8b56-2b4d03d7b657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.521883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbead26-beed-4aa8-8b56-2b4d03d7b657 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.544369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 987/1968] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:43 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:43.548861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e8c5-9aaf-49af-a19e-83fa0a915d63 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.559283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e8c5-9aaf-49af-a19e-83fa0a915d63 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.564445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c780e0c-e948-4fd1-9f01-bc486355e4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.565051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c780e0c-e948-4fd1-9f01-bc486355e4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.565331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c780e0c-e948-4fd1-9f01-bc486355e4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.565635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c780e0c-e948-4fd1-9f01-bc486355e4a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.566323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c780e0c-e948-4fd1-9f01-bc486355e4a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.567900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c780e0c-e948-4fd1-9f01-bc486355e4a0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.573075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:43.579847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 982/1969] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:43 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:43.583780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbead26-beed-4aa8-8b56-2b4d03d7b657 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.594000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbead26-beed-4aa8-8b56-2b4d03d7b657 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.599786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d878f0-e73f-474b-b339-a4e501220a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.600464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d878f0-e73f-474b-b339-a4e501220a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.600740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d878f0-e73f-474b-b339-a4e501220a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.601019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d878f0-e73f-474b-b339-a4e501220a91 admin admin] PATH_INFO: `/v3/projects/f9c7d251b149414589473fe7ca3b9d6b/users/f9afbc6e3b69482aa7590c4a93d78a96/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.616210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d878f0-e73f-474b-b339-a4e501220a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c7d251b149414589473fe7ca3b9d6b, user_id=f9afbc6e3b69482aa7590c4a93d78a96, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.618971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d878f0-e73f-474b-b339-a4e501220a91 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.621152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:43.633641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 988/1970] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:43 2022] PUT /identity/v3/projects/f9c7d251b149414589473fe7ca3b9d6b/users/f9afbc6e3b69482aa7590c4a93d78a96/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:43.637842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c780e0c-e948-4fd1-9f01-bc486355e4a0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.648641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c780e0c-e948-4fd1-9f01-bc486355e4a0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.706499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643dd36-42a4-4118-87ae-0605ed4f4c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.707122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643dd36-42a4-4118-87ae-0605ed4f4c0c admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.707405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643dd36-42a4-4118-87ae-0605ed4f4c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.707680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643dd36-42a4-4118-87ae-0605ed4f4c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.708371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643dd36-42a4-4118-87ae-0605ed4f4c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.710045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643dd36-42a4-4118-87ae-0605ed4f4c0c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.720376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 983/1971] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:43 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:43.724520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d878f0-e73f-474b-b339-a4e501220a91 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.741386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d878f0-e73f-474b-b339-a4e501220a91 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.746193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274be40-585f-4677-aff5-e2794bbdcc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.747003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274be40-585f-4677-aff5-e2794bbdcc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.747284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274be40-585f-4677-aff5-e2794bbdcc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.747809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274be40-585f-4677-aff5-e2794bbdcc47 admin admin] PATH_INFO: `/v3/projects/f9c7d251b149414589473fe7ca3b9d6b/users/f9afbc6e3b69482aa7590c4a93d78a96/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.749793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274be40-585f-4677-aff5-e2794bbdcc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c7d251b149414589473fe7ca3b9d6b, user_id=f9afbc6e3b69482aa7590c4a93d78a96, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.752849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274be40-585f-4677-aff5-e2794bbdcc47 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.763325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 989/1972] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:43 2022] PUT /identity/v3/projects/f9c7d251b149414589473fe7ca3b9d6b/users/f9afbc6e3b69482aa7590c4a93d78a96/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:43.773646 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be7d23-6bd4-460a-8210-7c2f177af915 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.773921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be7d23-6bd4-460a-8210-7c2f177af915 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.774198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be7d23-6bd4-460a-8210-7c2f177af915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.803916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-09be7d23-6bd4-460a-8210-7c2f177af915 None None] MFA Rules not processed for user `f9afbc6e3b69482aa7590c4a93d78a96`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:43.805134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-09be7d23-6bd4-460a-8210-7c2f177af915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:43.877701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 984/1973] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:43.890874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274be40-585f-4677-aff5-e2794bbdcc47 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:43.946972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274be40-585f-4677-aff5-e2794bbdcc47 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:43.950558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b29b6f-c5aa-4f98-9615-31f6e1f62aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:43.951174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b29b6f-c5aa-4f98-9615-31f6e1f62aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:43.951452 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b29b6f-c5aa-4f98-9615-31f6e1f62aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:43.951723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b29b6f-c5aa-4f98-9615-31f6e1f62aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:43.961996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b29b6f-c5aa-4f98-9615-31f6e1f62aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:43.963442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b29b6f-c5aa-4f98-9615-31f6e1f62aa6 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:43.973242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 990/1974] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:44.059805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be7d23-6bd4-460a-8210-7c2f177af915 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:44.061668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-09be7d23-6bd4-460a-8210-7c2f177af915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:44.073874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be7d23-6bd4-460a-8210-7c2f177af915 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:44.078801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb1515-4265-4159-9cb2-db97e18354f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:44.078801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb1515-4265-4159-9cb2-db97e18354f1 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:44.079154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb1515-4265-4159-9cb2-db97e18354f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:44.079958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb1515-4265-4159-9cb2-db97e18354f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:44.155316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb1515-4265-4159-9cb2-db97e18354f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:44.156805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb1515-4265-4159-9cb2-db97e18354f1 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:44.166631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 985/1975] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:49.858818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa396-1a5a-4391-b67f-c46ef5b1efe9 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:49.859327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa396-1a5a-4391-b67f-c46ef5b1efe9 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:49.859419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa396-1a5a-4391-b67f-c46ef5b1efe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:49.879225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-727fa396-1a5a-4391-b67f-c46ef5b1efe9 None None] MFA Rules not processed for user `9542521c42b34769a236eb6f268c5358`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:49.880669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-727fa396-1a5a-4391-b67f-c46ef5b1efe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:49.937298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 991/1976] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:51.040414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb1515-4265-4159-9cb2-db97e18354f1 service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:51.052514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb1515-4265-4159-9cb2-db97e18354f1 service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:51.062460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7670327-7316-4666-9e17-0af2e0ab1dfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:51.064705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7670327-7316-4666-9e17-0af2e0ab1dfc service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:51.064994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7670327-7316-4666-9e17-0af2e0ab1dfc service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:51.065268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7670327-7316-4666-9e17-0af2e0ab1dfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:51.069580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b7670327-7316-4666-9e17-0af2e0ab1dfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:51.097563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:51.159641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7670327-7316-4666-9e17-0af2e0ab1dfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:51.161152 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7670327-7316-4666-9e17-0af2e0ab1dfc service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:51.177024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 986/1977] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:51 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:51.220485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fa396-1a5a-4391-b67f-c46ef5b1efe9 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:51.222201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-727fa396-1a5a-4391-b67f-c46ef5b1efe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:51.336340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fa396-1a5a-4391-b67f-c46ef5b1efe9 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:51.348440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:51.350005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:51.350314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:51.350596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] PATH_INFO: `/v3/users/8c22e86703a14a3b986f42f5bd870808` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:51.352030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c22e86703a14a3b986f42f5bd870808)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:51.354464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:51.411006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:51.436016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c22e86703a14a3b986f42f5bd870808'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:51.446972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c22e86703a14a3b986f42f5bd870808'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:51.447833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c22e86703a14a3b986f42f5bd870808'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:51.462361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c22e86703a14a3b986f42f5bd870808'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:51.463145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c22e86703a14a3b986f42f5bd870808'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:51.482983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 992/1978] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:51 2022] DELETE /identity/v3/users/8c22e86703a14a3b986f42f5bd870808 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:51.488467 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde340e7-916e-4c3d-bc11-264ab6b2a43e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:51.489468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde340e7-916e-4c3d-bc11-264ab6b2a43e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:51.489675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde340e7-916e-4c3d-bc11-264ab6b2a43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:51.514883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-fde340e7-916e-4c3d-bc11-264ab6b2a43e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:51.516426 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fde340e7-916e-4c3d-bc11-264ab6b2a43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:51.599537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 987/1979] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:51.609563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:51.611727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:51.701510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d3679-325e-4f9b-9695-7952dd5dc9e3 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:51.706174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45124ab-f41b-4dce-ad34-9fb6c760b498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:51.706798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45124ab-f41b-4dce-ad34-9fb6c760b498 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:51.707191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45124ab-f41b-4dce-ad34-9fb6c760b498 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:51.708037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45124ab-f41b-4dce-ad34-9fb6c760b498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:51.720288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45124ab-f41b-4dce-ad34-9fb6c760b498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:51.721935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45124ab-f41b-4dce-ad34-9fb6c760b498 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:51.736053 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 993/1980] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:21:52.075433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde340e7-916e-4c3d-bc11-264ab6b2a43e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.077572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fde340e7-916e-4c3d-bc11-264ab6b2a43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:52.097189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde340e7-916e-4c3d-bc11-264ab6b2a43e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:52.108706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:52.109365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.109708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.110011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] PATH_INFO: `/v3/projects/43d1fd029f3a4963824e93f57cb97809` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.111204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d1fd029f3a4963824e93f57cb97809)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:52.113173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:52.163367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d1fd029f3a4963824e93f57cb97809', 'request_id': 'req-98b18030-963f-450a-8fb4-8f2becc914fc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.168935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d1fd029f3a4963824e93f57cb97809', 'request_id': 'req-98b18030-963f-450a-8fb4-8f2becc914fc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.176034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d1fd029f3a4963824e93f57cb97809', 'request_id': 'req-98b18030-963f-450a-8fb4-8f2becc914fc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.187082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] The token cache is being invalidate because project 43d1fd029f3a4963824e93f57cb97809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:52.187363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.188515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98b18030-963f-450a-8fb4-8f2becc914fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.190243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 988/1981] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:52 2022] DELETE /identity/v3/projects/43d1fd029f3a4963824e93f57cb97809 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:52.195136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47494e3b-6ade-4ffd-86e9-ea3fa4938939 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.195795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47494e3b-6ade-4ffd-86e9-ea3fa4938939 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.196256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47494e3b-6ade-4ffd-86e9-ea3fa4938939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.218890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-47494e3b-6ade-4ffd-86e9-ea3fa4938939 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:52.220153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-47494e3b-6ade-4ffd-86e9-ea3fa4938939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:52.280117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 994/1982] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:52.289849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f071051-3165-41ca-b709-ab2e539da288 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.290139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f071051-3165-41ca-b709-ab2e539da288 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.290484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f071051-3165-41ca-b709-ab2e539da288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.312151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4f071051-3165-41ca-b709-ab2e539da288 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:52.313799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4f071051-3165-41ca-b709-ab2e539da288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:52.321881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 989/1983] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:52.326261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47494e3b-6ade-4ffd-86e9-ea3fa4938939 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.336796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47494e3b-6ade-4ffd-86e9-ea3fa4938939 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:52.341516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:52.342121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.342403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.342681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.343369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:52.343674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:52.346060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:52.360019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 995/1984] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:52.364209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f071051-3165-41ca-b709-ab2e539da288 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.374231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f071051-3165-41ca-b709-ab2e539da288 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:52.379124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cdc23-3088-4219-9364-03f0bab48f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:52.379830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cdc23-3088-4219-9364-03f0bab48f88 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.380110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cdc23-3088-4219-9364-03f0bab48f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.380389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cdc23-3088-4219-9364-03f0bab48f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.381158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cdc23-3088-4219-9364-03f0bab48f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:52.383245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cdc23-3088-4219-9364-03f0bab48f88 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:52.419022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 990/1985] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:52 2022] POST /identity/v3/projects => generated 366 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:52.423401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.437315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d16ef-72e0-48be-8821-c8eb99587577 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:52.483785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:52.518665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:52.519295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.519577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.519856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.520541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:52.523083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:52.557703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 996/1986] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:52 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:52.558974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cdc23-3088-4219-9364-03f0bab48f88 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.588063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cdc23-3088-4219-9364-03f0bab48f88 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:52.593167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d718659-37fb-4014-9603-659f883c8063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:52.594823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d718659-37fb-4014-9603-659f883c8063 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.594960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d718659-37fb-4014-9603-659f883c8063 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.595932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d718659-37fb-4014-9603-659f883c8063 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.596956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d718659-37fb-4014-9603-659f883c8063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:52.598724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d718659-37fb-4014-9603-659f883c8063 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:52.612148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.614453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:52.621930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 991/1987] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:52 2022] GET /identity/v3/roles => generated 1427 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:52.627779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d718659-37fb-4014-9603-659f883c8063 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.635818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:52.636008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117a83d-ebcd-4273-9258-a4d740ab265d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:52.645326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d718659-37fb-4014-9603-659f883c8063 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:52.649087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:52.649813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.650095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.650367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] PATH_INFO: `/v3/users/7c75350e018a4b2f8b481be85dc88d9e` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.652806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c75350e018a4b2f8b481be85dc88d9e)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:52.658143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:52.660663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b2b3c-0fbe-4cfb-b4e3-db9d691c5111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:52.661511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b2b3c-0fbe-4cfb-b4e3-db9d691c5111 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.661880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b2b3c-0fbe-4cfb-b4e3-db9d691c5111 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.662196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b2b3c-0fbe-4cfb-b4e3-db9d691c5111 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.671986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b2b3c-0fbe-4cfb-b4e3-db9d691c5111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:52.674143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b2b3c-0fbe-4cfb-b4e3-db9d691c5111 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:52.690780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 992/1988] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:52 2022] GET /identity/v3/roles => generated 1427 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:52.691726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b2b3c-0fbe-4cfb-b4e3-db9d691c5111 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.704749 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b2b3c-0fbe-4cfb-b4e3-db9d691c5111 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:52.709933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9951e-6c01-4c68-9266-d0636e8d90f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:52.710623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9951e-6c01-4c68-9266-d0636e8d90f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.711623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9951e-6c01-4c68-9266-d0636e8d90f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.711884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9951e-6c01-4c68-9266-d0636e8d90f0 admin admin] PATH_INFO: `/v3/projects/077d1444b45e46d8943b4ecaa2523806/users/15f1d1302fe34c15bb962e39ecf13469/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.731386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9951e-6c01-4c68-9266-d0636e8d90f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=077d1444b45e46d8943b4ecaa2523806, user_id=15f1d1302fe34c15bb962e39ecf13469, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:52.737438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9951e-6c01-4c68-9266-d0636e8d90f0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:52.755502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c75350e018a4b2f8b481be85dc88d9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.761012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 993/1989] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:52 2022] PUT /identity/v3/projects/077d1444b45e46d8943b4ecaa2523806/users/15f1d1302fe34c15bb962e39ecf13469/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:52.772113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcf3f-5269-49b7-a877-e2148f61cf10 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.772307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c75350e018a4b2f8b481be85dc88d9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.772422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcf3f-5269-49b7-a877-e2148f61cf10 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.772661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcf3f-5269-49b7-a877-e2148f61cf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.773801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c75350e018a4b2f8b481be85dc88d9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.795313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c75350e018a4b2f8b481be85dc88d9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.796226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cf7475cd-76dd-42ec-91b4-e4710a1438dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c75350e018a4b2f8b481be85dc88d9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:52.819826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 997/1990] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:52 2022] DELETE /identity/v3/users/7c75350e018a4b2f8b481be85dc88d9e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:52.822882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-b71fcf3f-5269-49b7-a877-e2148f61cf10 None None] MFA Rules not processed for user `15f1d1302fe34c15bb962e39ecf13469`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:52.824416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b71fcf3f-5269-49b7-a877-e2148f61cf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:52.826907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631384f4-018c-4151-b6c4-0466cc209f19 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:52.827268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631384f4-018c-4151-b6c4-0466cc209f19 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:52.827718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631384f4-018c-4151-b6c4-0466cc209f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:52.850388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-631384f4-018c-4151-b6c4-0466cc209f19 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:52.851871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-631384f4-018c-4151-b6c4-0466cc209f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:52.922176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 994/1991] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:52.935760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fcf3f-5269-49b7-a877-e2148f61cf10 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.939399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b71fcf3f-5269-49b7-a877-e2148f61cf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:52.945384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 998/1992] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:52.958146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631384f4-018c-4151-b6c4-0466cc209f19 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:52.959527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-631384f4-018c-4151-b6c4-0466cc209f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:53.025669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:53.040200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fcf3f-5269-49b7-a877-e2148f61cf10 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:53.044182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:53.045047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:53.045591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:53.045934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:53.051208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631384f4-018c-4151-b6c4-0466cc209f19 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:53.055606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:53.056065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:53.056700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:53.056811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:53.057069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:53.057350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:53.066762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 995/1993] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:21:53.069598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:53.071073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:53.096292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 999/1994] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:21:53.893624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:53.897142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:53.919819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:53.921598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:53.938464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f8ec2-4551-4e7d-bf71-aa953081f790 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:53.945602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:53.946438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:53.947455 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:53.949803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] PATH_INFO: `/v3/users/3f72e96ab3a94bf7b27e9baa93e36371` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:53.952134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f72e96ab3a94bf7b27e9baa93e36371)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:53.958013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:53.966239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a77fd-a3dd-40f4-ae26-8da5b01d48fb service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:53.974701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:53.975879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:53.976328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:53.976998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] PATH_INFO: `/v3/projects/41a48a8a5330451788a9de67e1477f13` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:53.978642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a48a8a5330451788a9de67e1477f13)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:53.982249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:54.041114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f72e96ab3a94bf7b27e9baa93e36371'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.061930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f72e96ab3a94bf7b27e9baa93e36371'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.063834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f72e96ab3a94bf7b27e9baa93e36371'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.079943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a48a8a5330451788a9de67e1477f13', 'request_id': 'req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.093698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:54.133662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a48a8a5330451788a9de67e1477f13', 'request_id': 'req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.136838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f72e96ab3a94bf7b27e9baa93e36371'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.137947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-482fdb2b-5263-4c26-b8af-3dbe334266fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f72e96ab3a94bf7b27e9baa93e36371'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.158626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1000/1995] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:53 2022] DELETE /identity/v3/users/3f72e96ab3a94bf7b27e9baa93e36371 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:54.160662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f9065-04cf-43c1-b10e-62877f1d8f56 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:54.161007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f9065-04cf-43c1-b10e-62877f1d8f56 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:54.161630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f9065-04cf-43c1-b10e-62877f1d8f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:54.176430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a48a8a5330451788a9de67e1477f13', 'request_id': 'req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.206288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] The token cache is being invalidate because project 41a48a8a5330451788a9de67e1477f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:54.206829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.207649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.209357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-214f9065-04cf-43c1-b10e-62877f1d8f56 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:54.210487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-214f9065-04cf-43c1-b10e-62877f1d8f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:54.217082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 996/1996] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:53 2022] DELETE /identity/v3/projects/41a48a8a5330451788a9de67e1477f13 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:54.222368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:54.225554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:54.325607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1001/1997] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:54.347230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f9065-04cf-43c1-b10e-62877f1d8f56 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:54.350311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-214f9065-04cf-43c1-b10e-62877f1d8f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:54.355983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c2cb-a7f3-4b75-a851-a7f1bfc4c6db admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:54.362445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:54.362623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:54.363114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:54.363475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] PATH_INFO: `/v3/users/f9afbc6e3b69482aa7590c4a93d78a96` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:54.365386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9afbc6e3b69482aa7590c4a93d78a96)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:54.368563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:54.450225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9afbc6e3b69482aa7590c4a93d78a96'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.463721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9afbc6e3b69482aa7590c4a93d78a96'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.474934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f9065-04cf-43c1-b10e-62877f1d8f56 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:54.483474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:54.484241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:54.484621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:54.485049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:54.486251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9afbc6e3b69482aa7590c4a93d78a96'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.486976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9afbc6e3b69482aa7590c4a93d78a96'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.499406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:54.503310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9afbc6e3b69482aa7590c4a93d78a96'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:54.503466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:54.508682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:54.513188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 997/1998] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:54 2022] DELETE /identity/v3/users/f9afbc6e3b69482aa7590c4a93d78a96 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:54.523662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1002/1999] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:21:55.514794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:55.518026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:55.571639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:55.573816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:55.596459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb643ce-9bb3-4fff-b6c5-a80d46e2e909 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:55.701823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:55.702488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:55.702772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:55.703048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] PATH_INFO: `/v3/projects/dff1d0621a374c15ad868769546fc57a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:55.704336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff1d0621a374c15ad868769546fc57a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:55.706353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:55.709775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053138d-b800-4a24-b4d3-59419d513829 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:55.721043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:55.722255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:55.722747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:55.723230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] PATH_INFO: `/v3/projects/f9c7d251b149414589473fe7ca3b9d6b` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:55.724662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c7d251b149414589473fe7ca3b9d6b)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:55.726949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:55.730512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:55.771615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff1d0621a374c15ad868769546fc57a', 'request_id': 'req-71baf784-3cfc-444f-9930-67dc3dd473e4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.785894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff1d0621a374c15ad868769546fc57a', 'request_id': 'req-71baf784-3cfc-444f-9930-67dc3dd473e4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.794531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff1d0621a374c15ad868769546fc57a', 'request_id': 'req-71baf784-3cfc-444f-9930-67dc3dd473e4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.800290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c7d251b149414589473fe7ca3b9d6b', 'request_id': 'req-f84b12ee-4d50-46b7-adff-1c00de278701'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.808199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c7d251b149414589473fe7ca3b9d6b', 'request_id': 'req-f84b12ee-4d50-46b7-adff-1c00de278701'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.825142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c7d251b149414589473fe7ca3b9d6b', 'request_id': 'req-f84b12ee-4d50-46b7-adff-1c00de278701'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.826240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] The token cache is being invalidate because project dff1d0621a374c15ad868769546fc57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:55.826571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.827254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.832327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1003/2000] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:55 2022] DELETE /identity/v3/projects/dff1d0621a374c15ad868769546fc57a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:55.833951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:55.837198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:55.844343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] The token cache is being invalidate because project f9c7d251b149414589473fe7ca3b9d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:55.845010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.846281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f84b12ee-4d50-46b7-adff-1c00de278701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:55.848077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 998/2001] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:55 2022] DELETE /identity/v3/projects/f9c7d251b149414589473fe7ca3b9d6b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:55.853889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fb338-2154-4b24-a569-105c5325e846 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:55.854451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fb338-2154-4b24-a569-105c5325e846 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:55.855009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fb338-2154-4b24-a569-105c5325e846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:55.880617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-900fb338-2154-4b24-a569-105c5325e846 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:55.882444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-900fb338-2154-4b24-a569-105c5325e846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:56.014755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf784-3cfc-444f-9930-67dc3dd473e4 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.024219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:56.027361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.027878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.028338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] PATH_INFO: `/v3/users/bab0f2cb28324dc5b9915f89d1f3748a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.030526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bab0f2cb28324dc5b9915f89d1f3748a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:56.033541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 999/2002] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:56.034147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:56.042075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14bc6-dc3d-44fc-9aaa-8b609b1135bc None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.042982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14bc6-dc3d-44fc-9aaa-8b609b1135bc None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.043644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14bc6-dc3d-44fc-9aaa-8b609b1135bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.088083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6bf14bc6-dc3d-44fc-9aaa-8b609b1135bc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:56.090603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6bf14bc6-dc3d-44fc-9aaa-8b609b1135bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:56.120599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:56.124905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1000/2003] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:56.133208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf14bc6-dc3d-44fc-9aaa-8b609b1135bc None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:56.156773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bab0f2cb28324dc5b9915f89d1f3748a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.158969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf14bc6-dc3d-44fc-9aaa-8b609b1135bc None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.168626 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:56.170111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.170852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.171653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.173954 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:56.174899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:56.179017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:56.180278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bab0f2cb28324dc5b9915f89d1f3748a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.181138 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bab0f2cb28324dc5b9915f89d1f3748a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.204928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bab0f2cb28324dc5b9915f89d1f3748a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.205717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bab0f2cb28324dc5b9915f89d1f3748a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.209740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1001/2004] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:56.211459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:56.214326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:56.221972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1004/2005] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:55 2022] DELETE /identity/v3/users/bab0f2cb28324dc5b9915f89d1f3748a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:56.248371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5b1f0-c24d-4af9-bab2-07a31caa4f3a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.323974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64832aa-70a2-425f-87b0-2208cc6e48ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:56.324619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64832aa-70a2-425f-87b0-2208cc6e48ab admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.324937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64832aa-70a2-425f-87b0-2208cc6e48ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.325207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64832aa-70a2-425f-87b0-2208cc6e48ab admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.325927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64832aa-70a2-425f-87b0-2208cc6e48ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:56.327877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64832aa-70a2-425f-87b0-2208cc6e48ab admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:56.364643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:56.375688 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1002/2006] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:56 2022] POST /identity/v3/projects => generated 384 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:21:56.379959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:56.409950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d034c4-6819-462a-810c-2d58feddceb1 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.533102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1750671-b37f-450e-841a-e41e8e076da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:56.534099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1750671-b37f-450e-841a-e41e8e076da5 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.534689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1750671-b37f-450e-841a-e41e8e076da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.535279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1750671-b37f-450e-841a-e41e8e076da5 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.536097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1750671-b37f-450e-841a-e41e8e076da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:56.538233 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1750671-b37f-450e-841a-e41e8e076da5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:56.586874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1005/2007] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:56 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:56.591089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64832aa-70a2-425f-87b0-2208cc6e48ab admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:56.602609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64832aa-70a2-425f-87b0-2208cc6e48ab admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.611723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:56.612373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.612682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.612959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.613699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:56.616658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:56.631671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1003/2008] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:56 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:21:56.632663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1750671-b37f-450e-841a-e41e8e076da5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:56.644102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1750671-b37f-450e-841a-e41e8e076da5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.646708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:56.649599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:56.660993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c5a7e-2c2e-477e-aec3-b69e41d9bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:56.661676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5a7e-2c2e-477e-aec3-b69e41d9bab4 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.661963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5a7e-2c2e-477e-aec3-b69e41d9bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.662745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c5a7e-2c2e-477e-aec3-b69e41d9bab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.663065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c5a7e-2c2e-477e-aec3-b69e41d9bab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:56.664655 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c5a7e-2c2e-477e-aec3-b69e41d9bab4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:56.677212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5596d-0500-4427-a52b-9d48bc1021e0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.684002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1006/2009] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:56 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:56.686520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:56.688623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c5a7e-2c2e-477e-aec3-b69e41d9bab4 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:56.689164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.689521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.691801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] PATH_INFO: `/v3/projects/57d3bc56165b41b3afd78f6f979b03c6` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.691801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d3bc56165b41b3afd78f6f979b03c6)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:56.694316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:56.723008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c5a7e-2c2e-477e-aec3-b69e41d9bab4 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.738402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60306ae7-013a-46a2-8ba7-00cf867a03f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:56.739104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60306ae7-013a-46a2-8ba7-00cf867a03f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.739457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60306ae7-013a-46a2-8ba7-00cf867a03f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.739737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60306ae7-013a-46a2-8ba7-00cf867a03f3 admin admin] PATH_INFO: `/v3/projects/5c720256f0354d909d25d69fe3140de0/users/1d4a6cde03b8442fbb402303dc372390/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.773555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60306ae7-013a-46a2-8ba7-00cf867a03f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c720256f0354d909d25d69fe3140de0, user_id=1d4a6cde03b8442fbb402303dc372390, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:56.775950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d3bc56165b41b3afd78f6f979b03c6', 'request_id': 'req-c1206084-c259-4e24-9424-88cb634d68b1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.776353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60306ae7-013a-46a2-8ba7-00cf867a03f3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:56.784355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d3bc56165b41b3afd78f6f979b03c6', 'request_id': 'req-c1206084-c259-4e24-9424-88cb634d68b1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.821885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1007/2010] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:56 2022] PUT /identity/v3/projects/5c720256f0354d909d25d69fe3140de0/users/1d4a6cde03b8442fbb402303dc372390/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:56.828698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d3bc56165b41b3afd78f6f979b03c6', 'request_id': 'req-c1206084-c259-4e24-9424-88cb634d68b1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.831880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b06fa-ea57-4369-b92f-e38abf079e93 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:56.832443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b06fa-ea57-4369-b92f-e38abf079e93 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:56.833039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b06fa-ea57-4369-b92f-e38abf079e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:56.842674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] The token cache is being invalidate because project 57d3bc56165b41b3afd78f6f979b03c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:56.843075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.844006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:56.846668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1004/2011] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:56 2022] DELETE /identity/v3/projects/57d3bc56165b41b3afd78f6f979b03c6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:21:56.850696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:56.853204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:56.879459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0a8b06fa-ea57-4369-b92f-e38abf079e93 None None] MFA Rules not processed for user `1d4a6cde03b8442fbb402303dc372390`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:56.881242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0a8b06fa-ea57-4369-b92f-e38abf079e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:56.978912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1206084-c259-4e24-9424-88cb634d68b1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:56.983874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:57.026400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1008/2012] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:57.039621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b06fa-ea57-4369-b92f-e38abf079e93 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:57.041377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0a8b06fa-ea57-4369-b92f-e38abf079e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:57.084813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:57.086330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.086693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.087046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] PATH_INFO: `/v3/users/b3116a6f63d84f6e91e9f9c69468ae90` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.088896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3116a6f63d84f6e91e9f9c69468ae90)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:57.095098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:57.137927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:57.163463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b06fa-ea57-4369-b92f-e38abf079e93 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:57.168026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:57.168026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.168419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.168608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.180912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:57.185257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:57.187210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3116a6f63d84f6e91e9f9c69468ae90'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.198319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3116a6f63d84f6e91e9f9c69468ae90'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.221683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3116a6f63d84f6e91e9f9c69468ae90'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.222410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3116a6f63d84f6e91e9f9c69468ae90'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.225308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1009/2013] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:57.237015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-591a1cd6-97db-4e41-affd-2f402c139e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3116a6f63d84f6e91e9f9c69468ae90'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.244778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1005/2014] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:56 2022] DELETE /identity/v3/users/b3116a6f63d84f6e91e9f9c69468ae90 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:57.245440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:57.247642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:57.263015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff9957-abd3-4ec9-b257-7da5b249755a service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:57.334644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:57.335652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.335953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.336307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] PATH_INFO: `/v3/users/1d4a6cde03b8442fbb402303dc372390` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.338208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4a6cde03b8442fbb402303dc372390)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:57.340104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:57.399250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4a6cde03b8442fbb402303dc372390'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.408638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4a6cde03b8442fbb402303dc372390'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.409606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4a6cde03b8442fbb402303dc372390'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.431437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4a6cde03b8442fbb402303dc372390'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.432115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4a6cde03b8442fbb402303dc372390'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.447635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1010/2015] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:21:57 2022] DELETE /identity/v3/users/1d4a6cde03b8442fbb402303dc372390 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:57.451814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eb980-fdfc-487e-8d8b-ebfbb9b02f00 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.452106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eb980-fdfc-487e-8d8b-ebfbb9b02f00 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.452360 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eb980-fdfc-487e-8d8b-ebfbb9b02f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.473385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e76eb980-fdfc-487e-8d8b-ebfbb9b02f00 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:57.474678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e76eb980-fdfc-487e-8d8b-ebfbb9b02f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:57.545736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1006/2016] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:57.556600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:57.558191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:57.567030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76eb980-fdfc-487e-8d8b-ebfbb9b02f00 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:57.568669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e76eb980-fdfc-487e-8d8b-ebfbb9b02f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:57.576052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:57.583350 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76eb980-fdfc-487e-8d8b-ebfbb9b02f00 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:57.588742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:57.589395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.589702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.589973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] PATH_INFO: `/v3/projects/d113d20821bd4caca9802e0b3e6a08b8` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.591139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d113d20821bd4caca9802e0b3e6a08b8)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:57.593109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:57.635171 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f4b8-167d-45bf-bba5-3fdb7bc919fe admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:57.643430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd113d20821bd4caca9802e0b3e6a08b8', 'request_id': 'req-1cfc9538-9232-440b-8911-e109adc22c96'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.648916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd113d20821bd4caca9802e0b3e6a08b8', 'request_id': 'req-1cfc9538-9232-440b-8911-e109adc22c96'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.657054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd113d20821bd4caca9802e0b3e6a08b8', 'request_id': 'req-1cfc9538-9232-440b-8911-e109adc22c96'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.669452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] The token cache is being invalidate because project d113d20821bd4caca9802e0b3e6a08b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:57.669452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.670216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-1cfc9538-9232-440b-8911-e109adc22c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:57.671705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1007/2017] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:57 2022] DELETE /identity/v3/projects/d113d20821bd4caca9802e0b3e6a08b8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:57.680653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c149543-402d-4f2b-9f1f-a6573cbb3244 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.680939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c149543-402d-4f2b-9f1f-a6573cbb3244 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.681220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c149543-402d-4f2b-9f1f-a6573cbb3244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.689787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:57.691153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.691153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.691153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.692727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:57.701744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3c149543-402d-4f2b-9f1f-a6573cbb3244 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:57.703369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3c149543-402d-4f2b-9f1f-a6573cbb3244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:57.717600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:57.759720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:57.761165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:57.768670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1008/2018] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:57.770994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1011/2019] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:21:57.776864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163e0cd-29d7-41db-8470-301dc3faec83 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.777141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163e0cd-29d7-41db-8470-301dc3faec83 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.777434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163e0cd-29d7-41db-8470-301dc3faec83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.796530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f163e0cd-29d7-41db-8470-301dc3faec83 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:57.798531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f163e0cd-29d7-41db-8470-301dc3faec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:57.807053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1009/2020] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:57.811895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:57.825621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c086a-6cab-40a4-a660-946c4288716f service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:57.832263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:57.833841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.834353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.834664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.835440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:57.835831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:57.837491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:57.861696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1012/2021] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:21:57.866304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163e0cd-29d7-41db-8470-301dc3faec83 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:57.877432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163e0cd-29d7-41db-8470-301dc3faec83 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:57.883218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a20df-a134-40af-ae6e-3cf81f11c62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:57.883984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a20df-a134-40af-ae6e-3cf81f11c62e admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:57.884275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a20df-a134-40af-ae6e-3cf81f11c62e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:57.884715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a20df-a134-40af-ae6e-3cf81f11c62e admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:57.885317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a20df-a134-40af-ae6e-3cf81f11c62e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:57.887271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a20df-a134-40af-ae6e-3cf81f11c62e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:57.918457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1010/2022] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:57 2022] POST /identity/v3/projects => generated 376 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:57.929909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:57.946850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac8149-9d54-43b9-b52c-df0a99df5cbc admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.027347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.028299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.028611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.028886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.029591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.031589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.043755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:58.069299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1013/2023] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:57 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:58.073977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a20df-a134-40af-ae6e-3cf81f11c62e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.084468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.086456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:58.087596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a20df-a134-40af-ae6e-3cf81f11c62e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.093522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31df209-5c48-4354-905f-feaf7e5e8a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.095988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31df209-5c48-4354-905f-feaf7e5e8a98 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.095988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31df209-5c48-4354-905f-feaf7e5e8a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.095988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31df209-5c48-4354-905f-feaf7e5e8a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.095988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31df209-5c48-4354-905f-feaf7e5e8a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.110579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31df209-5c48-4354-905f-feaf7e5e8a98 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.120440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef7292-9564-49d9-a3db-46a99cc8a632 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.129442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.130454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.131086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.131595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] PATH_INFO: `/v3/projects/5c720256f0354d909d25d69fe3140de0` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.131930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1011/2024] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:58 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:58.133289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c720256f0354d909d25d69fe3140de0)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.135633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.136343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31df209-5c48-4354-905f-feaf7e5e8a98 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.158436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31df209-5c48-4354-905f-feaf7e5e8a98 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.165157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0839-ace0-4c32-83bf-2c4cf04180ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.166033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0839-ace0-4c32-83bf-2c4cf04180ce admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.166323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0839-ace0-4c32-83bf-2c4cf04180ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.166670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0839-ace0-4c32-83bf-2c4cf04180ce admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.167284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e0839-ace0-4c32-83bf-2c4cf04180ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.169754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e0839-ace0-4c32-83bf-2c4cf04180ce admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.184377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1012/2025] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:58 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:58.189762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0839-ace0-4c32-83bf-2c4cf04180ce admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.208361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0839-ace0-4c32-83bf-2c4cf04180ce admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.222949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c720256f0354d909d25d69fe3140de0', 'request_id': 'req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:58.259112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c720256f0354d909d25d69fe3140de0', 'request_id': 'req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:58.277263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c720256f0354d909d25d69fe3140de0', 'request_id': 'req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:58.292899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] The token cache is being invalidate because project 5c720256f0354d909d25d69fe3140de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:21:58.294058 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57784-2d51-4c70-8420-535b7e104c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.294393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:58.294960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57784-2d51-4c70-8420-535b7e104c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.295409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57784-2d51-4c70-8420-535b7e104c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.295639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5918fff4-7f9b-4f86-83cf-226f9ce6cfc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:21:58.295743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57784-2d51-4c70-8420-535b7e104c24 admin admin] PATH_INFO: `/v3/projects/e9f84ad3b15a414c9520b95806374a5e/users/0755af23dada46629a2e45240463bce4/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.299958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1014/2026] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:21:58 2022] DELETE /identity/v3/projects/5c720256f0354d909d25d69fe3140de0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:58.308109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a65b-87a8-41b9-be7f-ebd495d0fef6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.309047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a65b-87a8-41b9-be7f-ebd495d0fef6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.309600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a65b-87a8-41b9-be7f-ebd495d0fef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.333255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9cd0a65b-87a8-41b9-be7f-ebd495d0fef6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:58.335116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a57784-2d51-4c70-8420-535b7e104c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f84ad3b15a414c9520b95806374a5e, user_id=0755af23dada46629a2e45240463bce4, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.335422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9cd0a65b-87a8-41b9-be7f-ebd495d0fef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:58.339071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a57784-2d51-4c70-8420-535b7e104c24 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.347037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1015/2027] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:58.355151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c982ab7-ca8d-45a5-a093-dd33099630dd None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.355764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c982ab7-ca8d-45a5-a093-dd33099630dd None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.356336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c982ab7-ca8d-45a5-a093-dd33099630dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.369871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1013/2028] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:58 2022] PUT /identity/v3/projects/e9f84ad3b15a414c9520b95806374a5e/users/0755af23dada46629a2e45240463bce4/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:58.373481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5746373-403a-4bf8-a046-efc8b75a61ab None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.373775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5746373-403a-4bf8-a046-efc8b75a61ab None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.374050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5746373-403a-4bf8-a046-efc8b75a61ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.386791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7c982ab7-ca8d-45a5-a093-dd33099630dd None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:58.388044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7c982ab7-ca8d-45a5-a093-dd33099630dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:58.428347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e5746373-403a-4bf8-a046-efc8b75a61ab None None] MFA Rules not processed for user `0755af23dada46629a2e45240463bce4`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:58.429708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e5746373-403a-4bf8-a046-efc8b75a61ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:58.471298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1016/2029] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:58.476042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c982ab7-ca8d-45a5-a093-dd33099630dd None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.477167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:58.489072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c982ab7-ca8d-45a5-a093-dd33099630dd None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.493915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.494529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.494804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.495196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.495894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:21:58.496193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.498938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.511179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1014/2030] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:58.515174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1017/2031] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:21:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:21:58.515322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5746373-403a-4bf8-a046-efc8b75a61ab None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.516509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.516920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e5746373-403a-4bf8-a046-efc8b75a61ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:58.527515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230ac6-ab18-410f-8708-f0db1fea0791 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.532724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bef53c-2227-4ef5-bbf0-d797f6037be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.533386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef53c-2227-4ef5-bbf0-d797f6037be1 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.533712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef53c-2227-4ef5-bbf0-d797f6037be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.534577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef53c-2227-4ef5-bbf0-d797f6037be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.534707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bef53c-2227-4ef5-bbf0-d797f6037be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.535584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5746373-403a-4bf8-a046-efc8b75a61ab None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.536551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bef53c-2227-4ef5-bbf0-d797f6037be1 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.542885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8c311-4c8b-400e-b4f0-cf34aeaf64ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.543524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8c311-4c8b-400e-b4f0-cf34aeaf64ce admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.543807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8c311-4c8b-400e-b4f0-cf34aeaf64ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.544082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8c311-4c8b-400e-b4f0-cf34aeaf64ce admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.544812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8c311-4c8b-400e-b4f0-cf34aeaf64ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.547786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8c311-4c8b-400e-b4f0-cf34aeaf64ce admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.573731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1018/2032] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/projects => generated 384 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:21:58.574845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bef53c-2227-4ef5-bbf0-d797f6037be1 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.590689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1015/2033] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/projects => generated 378 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:58.591881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8c311-4c8b-400e-b4f0-cf34aeaf64ce admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.635123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:58.661659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bef53c-2227-4ef5-bbf0-d797f6037be1 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.688482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8c311-4c8b-400e-b4f0-cf34aeaf64ce admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.694745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0e380-ed1f-4fc9-a374-cbf50f9c8af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.696011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0e380-ed1f-4fc9-a374-cbf50f9c8af4 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.696514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0e380-ed1f-4fc9-a374-cbf50f9c8af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.697021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0e380-ed1f-4fc9-a374-cbf50f9c8af4 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.698021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0e380-ed1f-4fc9-a374-cbf50f9c8af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.701451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0e380-ed1f-4fc9-a374-cbf50f9c8af4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.727412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bff046-ae96-4cc7-89e6-790414fdd0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.728288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff046-ae96-4cc7-89e6-790414fdd0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.728634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff046-ae96-4cc7-89e6-790414fdd0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.728969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff046-ae96-4cc7-89e6-790414fdd0b4 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.729806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bff046-ae96-4cc7-89e6-790414fdd0b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.735627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1016/2034] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:21:58.735885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bff046-ae96-4cc7-89e6-790414fdd0b4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.737658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0e380-ed1f-4fc9-a374-cbf50f9c8af4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.751880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0e380-ed1f-4fc9-a374-cbf50f9c8af4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.758216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb59bcb-b593-44e1-9b2d-f537073530fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.759165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb59bcb-b593-44e1-9b2d-f537073530fd admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.759636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb59bcb-b593-44e1-9b2d-f537073530fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.760158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb59bcb-b593-44e1-9b2d-f537073530fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.763243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1019/2035] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:58.763330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb59bcb-b593-44e1-9b2d-f537073530fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.764393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bff046-ae96-4cc7-89e6-790414fdd0b4 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.765400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb59bcb-b593-44e1-9b2d-f537073530fd admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.778835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bff046-ae96-4cc7-89e6-790414fdd0b4 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.782087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1017/2036] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:58 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:58.786553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb59bcb-b593-44e1-9b2d-f537073530fd admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.788108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5878137-ba43-4004-9228-c20d51aa90eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.788876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5878137-ba43-4004-9228-c20d51aa90eb admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.789218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5878137-ba43-4004-9228-c20d51aa90eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.789588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5878137-ba43-4004-9228-c20d51aa90eb admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.790362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5878137-ba43-4004-9228-c20d51aa90eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.792293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5878137-ba43-4004-9228-c20d51aa90eb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.801053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb59bcb-b593-44e1-9b2d-f537073530fd admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.809143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1020/2037] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:58 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:58.810408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49325e-9305-4dcb-87c0-9b501b7f61d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.810658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5878137-ba43-4004-9228-c20d51aa90eb admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.811615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49325e-9305-4dcb-87c0-9b501b7f61d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.812056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49325e-9305-4dcb-87c0-9b501b7f61d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.812868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49325e-9305-4dcb-87c0-9b501b7f61d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.813839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49325e-9305-4dcb-87c0-9b501b7f61d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.815656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49325e-9305-4dcb-87c0-9b501b7f61d0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.824693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5878137-ba43-4004-9228-c20d51aa90eb admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.833601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1018/2038] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:58 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:58.834761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08a53-922c-4213-b639-e49913066572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.835744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08a53-922c-4213-b639-e49913066572 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.835987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49325e-9305-4dcb-87c0-9b501b7f61d0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.836076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08a53-922c-4213-b639-e49913066572 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.836390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08a53-922c-4213-b639-e49913066572 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.837198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf08a53-922c-4213-b639-e49913066572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.839193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf08a53-922c-4213-b639-e49913066572 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.847303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49325e-9305-4dcb-87c0-9b501b7f61d0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.852279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01afab-fa71-46bf-9629-071e33598149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.853022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01afab-fa71-46bf-9629-071e33598149 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.853292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01afab-fa71-46bf-9629-071e33598149 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.853603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01afab-fa71-46bf-9629-071e33598149 admin admin] PATH_INFO: `/v3/projects/543e3c6bbad64df68dac4cc6d125956b/users/2200d1c0ef21454a89f2bd377b36f7ac/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.854496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1021/2039] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:58 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:58.857916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08a53-922c-4213-b639-e49913066572 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:58.872165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08a53-922c-4213-b639-e49913066572 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:58.880507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c9665-344b-4d2f-af2e-b6825bbb0e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:58.881222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01afab-fa71-46bf-9629-071e33598149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=543e3c6bbad64df68dac4cc6d125956b, user_id=2200d1c0ef21454a89f2bd377b36f7ac, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.881551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9665-344b-4d2f-af2e-b6825bbb0e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.881955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9665-344b-4d2f-af2e-b6825bbb0e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.882341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9665-344b-4d2f-af2e-b6825bbb0e30 admin admin] PATH_INFO: `/v3/projects/934bdfb95b0d41aabb2b5d0944e6aa50/users/2b3dda189a134516ae0493973a617fac/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.884828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01afab-fa71-46bf-9629-071e33598149 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.901809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c9665-344b-4d2f-af2e-b6825bbb0e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934bdfb95b0d41aabb2b5d0944e6aa50, user_id=2b3dda189a134516ae0493973a617fac, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:58.904771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1019/2040] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:58 2022] PUT /identity/v3/projects/543e3c6bbad64df68dac4cc6d125956b/users/2200d1c0ef21454a89f2bd377b36f7ac/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:58.905217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c9665-344b-4d2f-af2e-b6825bbb0e30 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:58.916380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e097a-f137-4ebe-8eff-8ecdd6342083 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.916880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e097a-f137-4ebe-8eff-8ecdd6342083 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.917000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e097a-f137-4ebe-8eff-8ecdd6342083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.935357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1022/2041] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:58 2022] PUT /identity/v3/projects/934bdfb95b0d41aabb2b5d0944e6aa50/users/2b3dda189a134516ae0493973a617fac/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:58.945958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51225cfd-124b-455d-ac15-631e14227415 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:58.946588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51225cfd-124b-455d-ac15-631e14227415 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:58.946941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51225cfd-124b-455d-ac15-631e14227415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:58.983282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-743e097a-f137-4ebe-8eff-8ecdd6342083 None None] MFA Rules not processed for user `2200d1c0ef21454a89f2bd377b36f7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:58.985287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-743e097a-f137-4ebe-8eff-8ecdd6342083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:58.990320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-51225cfd-124b-455d-ac15-631e14227415 None None] MFA Rules not processed for user `2b3dda189a134516ae0493973a617fac`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:21:58.991894 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-51225cfd-124b-455d-ac15-631e14227415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:59.142286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:59.148211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1023/2042] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:21:59.162585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51225cfd-124b-455d-ac15-631e14227415 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:59.164470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-51225cfd-124b-455d-ac15-631e14227415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:21:59.169848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1020/2043] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:21:59.171159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e097a-f137-4ebe-8eff-8ecdd6342083 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:59.204113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:21:59.257729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51225cfd-124b-455d-ac15-631e14227415 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:59.267367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735556e2-314a-4d1b-905a-d8c082316479 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:59.269015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735556e2-314a-4d1b-905a-d8c082316479 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:59.269539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735556e2-314a-4d1b-905a-d8c082316479 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:59.270039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735556e2-314a-4d1b-905a-d8c082316479 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:59.288113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735556e2-314a-4d1b-905a-d8c082316479 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:59.289734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735556e2-314a-4d1b-905a-d8c082316479 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:59.303863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e097a-f137-4ebe-8eff-8ecdd6342083 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:59.316162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6965c61-02f8-4484-97e5-0d7cccbd7f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:59.317986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965c61-02f8-4484-97e5-0d7cccbd7f5a admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:59.318976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965c61-02f8-4484-97e5-0d7cccbd7f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:59.319431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965c61-02f8-4484-97e5-0d7cccbd7f5a admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:59.319958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1024/2044] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:21:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:21:59.320973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6965c61-02f8-4484-97e5-0d7cccbd7f5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:59.326456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6965c61-02f8-4484-97e5-0d7cccbd7f5a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:59.379882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1021/2045] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:21:59 2022] POST /identity/v3/projects => generated 378 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:59.383396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735556e2-314a-4d1b-905a-d8c082316479 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:59.396990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735556e2-314a-4d1b-905a-d8c082316479 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:59.506008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc4fca-3a9f-4167-bd87-ef1609be635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:59.506821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4fca-3a9f-4167-bd87-ef1609be635c admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:59.506978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4fca-3a9f-4167-bd87-ef1609be635c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:59.507262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4fca-3a9f-4167-bd87-ef1609be635c admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:59.507960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc4fca-3a9f-4167-bd87-ef1609be635c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:59.509948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc4fca-3a9f-4167-bd87-ef1609be635c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:59.556038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1025/2046] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:21:59 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:21:59.574747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6965c61-02f8-4484-97e5-0d7cccbd7f5a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:59.609655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6965c61-02f8-4484-97e5-0d7cccbd7f5a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:59.636475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a6caa-58bf-437c-b00f-bfa4762883d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:59.638643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a6caa-58bf-437c-b00f-bfa4762883d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:59.638643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a6caa-58bf-437c-b00f-bfa4762883d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:59.638643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a6caa-58bf-437c-b00f-bfa4762883d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:59.642263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a6caa-58bf-437c-b00f-bfa4762883d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:59.644165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a6caa-58bf-437c-b00f-bfa4762883d2 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:59.670965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1022/2047] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:59 2022] GET /identity/v3/roles => generated 1427 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:59.676787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc4fca-3a9f-4167-bd87-ef1609be635c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:59.690175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc4fca-3a9f-4167-bd87-ef1609be635c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:59.696416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82445337-e7be-41d9-8ca7-018c1ea1212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:59.697104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445337-e7be-41d9-8ca7-018c1ea1212e admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:59.697386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445337-e7be-41d9-8ca7-018c1ea1212e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:59.697767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445337-e7be-41d9-8ca7-018c1ea1212e admin admin] PATH_INFO: `/v3/projects/f51104d97af74faf86a72526c3997e0a/users/888b76a3f851401b8dc0bb89c925a693/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:59.730551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82445337-e7be-41d9-8ca7-018c1ea1212e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51104d97af74faf86a72526c3997e0a, user_id=888b76a3f851401b8dc0bb89c925a693, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:59.735781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82445337-e7be-41d9-8ca7-018c1ea1212e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:59.766585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1026/2048] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:59 2022] PUT /identity/v3/projects/f51104d97af74faf86a72526c3997e0a/users/888b76a3f851401b8dc0bb89c925a693/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:59.768148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a6caa-58bf-437c-b00f-bfa4762883d2 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:59.780689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a6caa-58bf-437c-b00f-bfa4762883d2 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:59.906902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fc51a-c090-4d33-b394-7aeab25f63f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:59.907526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fc51a-c090-4d33-b394-7aeab25f63f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:59.907881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fc51a-c090-4d33-b394-7aeab25f63f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:59.908071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fc51a-c090-4d33-b394-7aeab25f63f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:59.908777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fc51a-c090-4d33-b394-7aeab25f63f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:59.910639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fc51a-c090-4d33-b394-7aeab25f63f8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:59.923267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1023/2049] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:21:59 2022] GET /identity/v3/roles => generated 1427 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:21:59.927210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82445337-e7be-41d9-8ca7-018c1ea1212e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:21:59.945623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82445337-e7be-41d9-8ca7-018c1ea1212e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:21:59.951051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e5532-2a6c-4c5f-88b8-a8454273f220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:21:59.951735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e5532-2a6c-4c5f-88b8-a8454273f220 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:59.952018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e5532-2a6c-4c5f-88b8-a8454273f220 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:59.952295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e5532-2a6c-4c5f-88b8-a8454273f220 admin admin] PATH_INFO: `/v3/projects/f51104d97af74faf86a72526c3997e0a/users/888b76a3f851401b8dc0bb89c925a693/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:21:59.955005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e5532-2a6c-4c5f-88b8-a8454273f220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51104d97af74faf86a72526c3997e0a, user_id=888b76a3f851401b8dc0bb89c925a693, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:21:59.957590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e5532-2a6c-4c5f-88b8-a8454273f220 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:21:59.970854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1027/2050] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:21:59 2022] PUT /identity/v3/projects/f51104d97af74faf86a72526c3997e0a/users/888b76a3f851401b8dc0bb89c925a693/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:21:59.983982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9c39b-ee01-40d5-abc8-832e37bf11c3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:21:59.984491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9c39b-ee01-40d5-abc8-832e37bf11c3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:21:59.984994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9c39b-ee01-40d5-abc8-832e37bf11c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:00.016414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-37d9c39b-ee01-40d5-abc8-832e37bf11c3 None None] MFA Rules not processed for user `888b76a3f851401b8dc0bb89c925a693`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:00.017978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-37d9c39b-ee01-40d5-abc8-832e37bf11c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:00.071376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:00.095794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1024/2051] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:21:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:00.108019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e5532-2a6c-4c5f-88b8-a8454273f220 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:00.166758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e5532-2a6c-4c5f-88b8-a8454273f220 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:00.170471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39934f21-65d1-484d-ad0e-28aff107e1ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:00.171096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39934f21-65d1-484d-ad0e-28aff107e1ac service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:00.171378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39934f21-65d1-484d-ad0e-28aff107e1ac service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:00.172352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39934f21-65d1-484d-ad0e-28aff107e1ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:00.200969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:00.235052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39934f21-65d1-484d-ad0e-28aff107e1ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:00.236569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39934f21-65d1-484d-ad0e-28aff107e1ac service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:00.246092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1028/2052] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:01.860567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9c39b-ee01-40d5-abc8-832e37bf11c3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:01.862806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-37d9c39b-ee01-40d5-abc8-832e37bf11c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:01.886479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9c39b-ee01-40d5-abc8-832e37bf11c3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:01.891099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf8121-013d-4846-bf83-6a0ab2148d1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:01.892090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf8121-013d-4846-bf83-6a0ab2148d1d service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:01.892515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf8121-013d-4846-bf83-6a0ab2148d1d service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:01.892880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf8121-013d-4846-bf83-6a0ab2148d1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:01.995533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf8121-013d-4846-bf83-6a0ab2148d1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:01.997512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf8121-013d-4846-bf83-6a0ab2148d1d service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:02.023155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1025/2053] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:22:03.411530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597566b-74c3-4144-9422-4979c609c73f None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:03.412032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597566b-74c3-4144-9422-4979c609c73f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:03.412204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597566b-74c3-4144-9422-4979c609c73f None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:03.413547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1029/2054] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:22:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:22:03.441730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ad066-439d-4891-91d8-72209e96abe6 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:03.442320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ad066-439d-4891-91d8-72209e96abe6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:03.442489 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ad066-439d-4891-91d8-72209e96abe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:03.462812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-196ad066-439d-4891-91d8-72209e96abe6 None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:03.464102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-196ad066-439d-4891-91d8-72209e96abe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:03.519448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597566b-74c3-4144-9422-4979c609c73f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:03.521871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6597566b-74c3-4144-9422-4979c609c73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:03.559311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1026/2055] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:22:03 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:03.579522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ad066-439d-4891-91d8-72209e96abe6 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:03.582735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-196ad066-439d-4891-91d8-72209e96abe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:03.596846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ad066-439d-4891-91d8-72209e96abe6 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:03.611992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80567a8-3be6-4c7a-be13-9cbb144c3974 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:03.613906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80567a8-3be6-4c7a-be13-9cbb144c3974 service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:03.614257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80567a8-3be6-4c7a-be13-9cbb144c3974 service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:03.614590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80567a8-3be6-4c7a-be13-9cbb144c3974 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:03.625759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80567a8-3be6-4c7a-be13-9cbb144c3974 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:03.627394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80567a8-3be6-4c7a-be13-9cbb144c3974 service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:03.638333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1027/2056] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:03 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:03.645002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597566b-74c3-4144-9422-4979c609c73f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:03.650282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:03.650939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:03.651229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:03.651637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] PATH_INFO: `/v3/users/84b6e8d7c0f94bd5a3ad5bf90386b34c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:03.652907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b6e8d7c0f94bd5a3ad5bf90386b34c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:03.659082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:03.709031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b6e8d7c0f94bd5a3ad5bf90386b34c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:03.716844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b6e8d7c0f94bd5a3ad5bf90386b34c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:03.717562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b6e8d7c0f94bd5a3ad5bf90386b34c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:03.730831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b6e8d7c0f94bd5a3ad5bf90386b34c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:03.731592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cefca900-2ff8-4932-82ff-5beb17466a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b6e8d7c0f94bd5a3ad5bf90386b34c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:03.744702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1030/2057] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:22:03 2022] DELETE /identity/v3/users/84b6e8d7c0f94bd5a3ad5bf90386b34c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:22:03.925108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80567a8-3be6-4c7a-be13-9cbb144c3974 service placement] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:03.936083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80567a8-3be6-4c7a-be13-9cbb144c3974 service placement] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:03.996133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:03.997028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:03.997302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:03.997600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] PATH_INFO: `/v3/projects/721722d703af4087bf9bb3854966b482` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:03.998746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=721722d703af4087bf9bb3854966b482)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:04.001353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:04.012882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:04.047877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721722d703af4087bf9bb3854966b482', 'request_id': 'req-71f4548d-a5a6-45f1-8e63-0e43bebe5434'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:04.053201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721722d703af4087bf9bb3854966b482', 'request_id': 'req-71f4548d-a5a6-45f1-8e63-0e43bebe5434'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:04.060620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721722d703af4087bf9bb3854966b482', 'request_id': 'req-71f4548d-a5a6-45f1-8e63-0e43bebe5434'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:04.072548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] The token cache is being invalidate because project 721722d703af4087bf9bb3854966b482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:22:04.072838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:04.073515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-71f4548d-a5a6-45f1-8e63-0e43bebe5434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:04.075029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1028/2058] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:22:03 2022] DELETE /identity/v3/projects/721722d703af4087bf9bb3854966b482 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:04.080641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991102c1-21d2-41ca-bae2-bc1fe1c403bc None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.080931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991102c1-21d2-41ca-bae2-bc1fe1c403bc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.081234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991102c1-21d2-41ca-bae2-bc1fe1c403bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.101453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-991102c1-21d2-41ca-bae2-bc1fe1c403bc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:04.102592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-991102c1-21d2-41ca-bae2-bc1fe1c403bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:04.147337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:04.166713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1031/2059] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:22:04.174269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b186-0f7b-4603-bcac-b6b4ed3fb8bc None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.174546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b186-0f7b-4603-bcac-b6b4ed3fb8bc None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.174816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b186-0f7b-4603-bcac-b6b4ed3fb8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.194355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-1858b186-0f7b-4603-bcac-b6b4ed3fb8bc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:04.195548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-1858b186-0f7b-4603-bcac-b6b4ed3fb8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:04.203799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1029/2060] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:04.207845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991102c1-21d2-41ca-bae2-bc1fe1c403bc None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:04.217650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991102c1-21d2-41ca-bae2-bc1fe1c403bc None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:04.222514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:04.223123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.223402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.223674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.224365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:22:04.224730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:04.226368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:04.238027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1032/2061] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:22:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:22:04.242115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858b186-0f7b-4603-bcac-b6b4ed3fb8bc None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:04.252025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858b186-0f7b-4603-bcac-b6b4ed3fb8bc None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:04.257068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d99455-18f4-460e-82a6-d74ca017efa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:04.257719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d99455-18f4-460e-82a6-d74ca017efa6 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.258001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d99455-18f4-460e-82a6-d74ca017efa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.258279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d99455-18f4-460e-82a6-d74ca017efa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.258963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d99455-18f4-460e-82a6-d74ca017efa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:04.261037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d99455-18f4-460e-82a6-d74ca017efa6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:04.292495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1030/2062] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:22:04 2022] POST /identity/v3/projects => generated 366 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:04.293947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:04.307542 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83971461-8f47-4cd1-b3cb-3ea0878f063c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:04.366391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cee5-fa7d-4d51-8030-4de93a833a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:04.366949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cee5-fa7d-4d51-8030-4de93a833a4d admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.367232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cee5-fa7d-4d51-8030-4de93a833a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.367503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cee5-fa7d-4d51-8030-4de93a833a4d admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.368163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914cee5-fa7d-4d51-8030-4de93a833a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:04.370140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914cee5-fa7d-4d51-8030-4de93a833a4d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:04.391021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1033/2063] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:22:04 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:04.395047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d99455-18f4-460e-82a6-d74ca017efa6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:04.405544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d99455-18f4-460e-82a6-d74ca017efa6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:04.410464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c8627-76d9-4cf4-a601-dee53453c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:04.411071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c8627-76d9-4cf4-a601-dee53453c000 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.411348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c8627-76d9-4cf4-a601-dee53453c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.411616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c8627-76d9-4cf4-a601-dee53453c000 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.412332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c8627-76d9-4cf4-a601-dee53453c000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:04.413940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c8627-76d9-4cf4-a601-dee53453c000 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:04.424315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1031/2064] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:04 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:22:04.428368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cee5-fa7d-4d51-8030-4de93a833a4d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:04.439685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cee5-fa7d-4d51-8030-4de93a833a4d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:04.444744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2a5f6-a035-4fbf-91e8-b8c418e2ea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:04.445376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a5f6-a035-4fbf-91e8-b8c418e2ea3e admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.445694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a5f6-a035-4fbf-91e8-b8c418e2ea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.445990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a5f6-a035-4fbf-91e8-b8c418e2ea3e admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.446704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2a5f6-a035-4fbf-91e8-b8c418e2ea3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:04.448355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2a5f6-a035-4fbf-91e8-b8c418e2ea3e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:04.459171 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1034/2065] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:04 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:04.463218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c8627-76d9-4cf4-a601-dee53453c000 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:04.473892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c8627-76d9-4cf4-a601-dee53453c000 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:04.478844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:04.479506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.479787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.480065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] PATH_INFO: `/v3/projects/1cd0f39c1d50454e97caf9625760ab4a/users/2ededf6b87e34578bc0991e7a560d4ad/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.495342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd0f39c1d50454e97caf9625760ab4a, user_id=2ededf6b87e34578bc0991e7a560d4ad, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:04.498292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:04.511835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1032/2066] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:04 2022] PUT /identity/v3/projects/1cd0f39c1d50454e97caf9625760ab4a/users/2ededf6b87e34578bc0991e7a560d4ad/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:04.522408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4734fa3-2dc8-4116-b61d-fed2744d6dea None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.522698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4734fa3-2dc8-4116-b61d-fed2744d6dea None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.522975 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4734fa3-2dc8-4116-b61d-fed2744d6dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.550745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-c4734fa3-2dc8-4116-b61d-fed2744d6dea None None] MFA Rules not processed for user `2ededf6b87e34578bc0991e7a560d4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:04.551953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c4734fa3-2dc8-4116-b61d-fed2744d6dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:04.616335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1035/2067] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:04.627881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:04.629616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:04.689559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e22daa-9a59-4939-9cc9-84369c8604df admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:04.694228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:04.694936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:04.695225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:04.695493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:04.707242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:04.708771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:04.719115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1033/2068] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:05.927066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4734fa3-2dc8-4116-b61d-fed2744d6dea None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:05.928518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c4734fa3-2dc8-4116-b61d-fed2744d6dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:05.941151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4734fa3-2dc8-4116-b61d-fed2744d6dea None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:05.944731 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f144e4-f3a0-4ee3-8b61-a6feb0d88a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:05.945347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f144e4-f3a0-4ee3-8b61-a6feb0d88a11 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:05.945654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f144e4-f3a0-4ee3-8b61-a6feb0d88a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:05.945924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f144e4-f3a0-4ee3-8b61-a6feb0d88a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:05.947674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-58f144e4-f3a0-4ee3-8b61-a6feb0d88a11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:05.990226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:06.013360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f144e4-f3a0-4ee3-8b61-a6feb0d88a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:06.014855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f144e4-f3a0-4ee3-8b61-a6feb0d88a11 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:06.025560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1036/2069] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:07.572233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:07.573919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:07.619285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:07.643570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e0e16-ada2-404d-99d5-dde691a4a7a6 service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:07.648077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:07.648710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:07.648987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:07.649255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] PATH_INFO: `/v3/users/2b3dda189a134516ae0493973a617fac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:07.650449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3dda189a134516ae0493973a617fac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:07.652467 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:07.698605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3dda189a134516ae0493973a617fac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:07.708951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3dda189a134516ae0493973a617fac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:07.715632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3dda189a134516ae0493973a617fac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:07.716313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3dda189a134516ae0493973a617fac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:07.729124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3dda189a134516ae0493973a617fac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:07.735634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1034/2070] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:22:07 2022] DELETE /identity/v3/users/2b3dda189a134516ae0493973a617fac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:07.740110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b619-b882-4029-83ed-18d45c6d6777 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:07.740617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b619-b882-4029-83ed-18d45c6d6777 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:07.740710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b619-b882-4029-83ed-18d45c6d6777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:07.760383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6241b619-b882-4029-83ed-18d45c6d6777 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:07.761669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6241b619-b882-4029-83ed-18d45c6d6777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:07.825600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1037/2071] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:07.835398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:07.836805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:07.904039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb13e4-5809-4f7b-bcc9-797ba03cfece admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:07.909091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01fa57-2ebb-48c7-89eb-d6b13afd231e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:07.909746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01fa57-2ebb-48c7-89eb-d6b13afd231e service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:07.910025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01fa57-2ebb-48c7-89eb-d6b13afd231e service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:07.910298 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01fa57-2ebb-48c7-89eb-d6b13afd231e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:07.919970 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01fa57-2ebb-48c7-89eb-d6b13afd231e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:07.921996 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01fa57-2ebb-48c7-89eb-d6b13afd231e service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:07.932716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1035/2072] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:08.249773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241b619-b882-4029-83ed-18d45c6d6777 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:08.253862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6241b619-b882-4029-83ed-18d45c6d6777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:08.279261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241b619-b882-4029-83ed-18d45c6d6777 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:08.288643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:08.290426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:08.290871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:08.291603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] PATH_INFO: `/v3/projects/934bdfb95b0d41aabb2b5d0944e6aa50` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:08.292985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=934bdfb95b0d41aabb2b5d0944e6aa50)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:08.295383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:08.364387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934bdfb95b0d41aabb2b5d0944e6aa50', 'request_id': 'req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:08.375745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934bdfb95b0d41aabb2b5d0944e6aa50', 'request_id': 'req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:08.391625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934bdfb95b0d41aabb2b5d0944e6aa50', 'request_id': 'req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:08.413530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] The token cache is being invalidate because project 934bdfb95b0d41aabb2b5d0944e6aa50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:22:08.413530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:08.414071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5cfd4dc7-37fd-49db-993a-91f6e28df2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:08.421631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1038/2073] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:22:08 2022] DELETE /identity/v3/projects/934bdfb95b0d41aabb2b5d0944e6aa50 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:08.423167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b3541-6892-43a6-8d51-289641f18987 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:08.423461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b3541-6892-43a6-8d51-289641f18987 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:08.423735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b3541-6892-43a6-8d51-289641f18987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:08.461340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-352b3541-6892-43a6-8d51-289641f18987 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:08.464707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-352b3541-6892-43a6-8d51-289641f18987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:08.577629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1036/2074] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:08.583929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304824-ed4c-4d30-8dcb-750572b23318 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:08.584143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304824-ed4c-4d30-8dcb-750572b23318 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:08.584425 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304824-ed4c-4d30-8dcb-750572b23318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:08.633211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2e304824-ed4c-4d30-8dcb-750572b23318 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:08.639875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2e304824-ed4c-4d30-8dcb-750572b23318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:08.654824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1039/2075] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:08.665442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b3541-6892-43a6-8d51-289641f18987 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:08.670613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:08.678815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b3541-6892-43a6-8d51-289641f18987 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:08.692280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:08.693741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:08.694374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:08.694896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:08.696009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:22:08.696481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:08.698374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:08.713042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1037/2076] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:22:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:22:08.717425 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304824-ed4c-4d30-8dcb-750572b23318 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:08.734423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304824-ed4c-4d30-8dcb-750572b23318 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:08.739786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927fb6e-08a7-4112-bf3d-ffa8721557a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:08.740392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927fb6e-08a7-4112-bf3d-ffa8721557a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:08.741080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927fb6e-08a7-4112-bf3d-ffa8721557a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:08.741575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927fb6e-08a7-4112-bf3d-ffa8721557a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:08.742582 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927fb6e-08a7-4112-bf3d-ffa8721557a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:08.745722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927fb6e-08a7-4112-bf3d-ffa8721557a7 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:08.790127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1040/2077] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:22:08 2022] POST /identity/v3/projects => generated 370 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:08.791409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:08.809748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f9fd1-8748-4923-8f09-c2d672c02a63 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:08.897013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b64eb2-2e00-4a37-95f4-d69d1220f9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:08.897663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b64eb2-2e00-4a37-95f4-d69d1220f9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:08.897959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b64eb2-2e00-4a37-95f4-d69d1220f9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:08.899060 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b64eb2-2e00-4a37-95f4-d69d1220f9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:08.900081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b64eb2-2e00-4a37-95f4-d69d1220f9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:08.902020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b64eb2-2e00-4a37-95f4-d69d1220f9c8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:08.928493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1038/2078] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:22:08 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:22:08.929769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927fb6e-08a7-4112-bf3d-ffa8721557a7 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:08.940720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927fb6e-08a7-4112-bf3d-ffa8721557a7 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:08.945503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a88858-6bcf-461b-b0f9-5622a36e066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:08.946224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a88858-6bcf-461b-b0f9-5622a36e066f admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:08.946692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a88858-6bcf-461b-b0f9-5622a36e066f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:08.946960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a88858-6bcf-461b-b0f9-5622a36e066f admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:08.947798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a88858-6bcf-461b-b0f9-5622a36e066f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:08.949475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a88858-6bcf-461b-b0f9-5622a36e066f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:08.960950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1041/2079] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:08 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:08.965016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b64eb2-2e00-4a37-95f4-d69d1220f9c8 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:08.976881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b64eb2-2e00-4a37-95f4-d69d1220f9c8 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:08.982062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be67be-7615-4feb-8e71-1f691cdd61ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:08.982692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be67be-7615-4feb-8e71-1f691cdd61ef admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:08.982972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be67be-7615-4feb-8e71-1f691cdd61ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:08.983239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be67be-7615-4feb-8e71-1f691cdd61ef admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:08.983925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be67be-7615-4feb-8e71-1f691cdd61ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:08.985663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be67be-7615-4feb-8e71-1f691cdd61ef admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:09.001100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1039/2080] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:08 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:09.002519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a88858-6bcf-461b-b0f9-5622a36e066f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:09.013077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a88858-6bcf-461b-b0f9-5622a36e066f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:09.019388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:09.020046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:09.020326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:09.020593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] PATH_INFO: `/v3/projects/c5a29b3102eb43fd94bdd568781904a9/users/96449207f1a34ef9b5c433e147de0c5f/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:09.036719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a29b3102eb43fd94bdd568781904a9, user_id=96449207f1a34ef9b5c433e147de0c5f, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:09.039459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:09.043114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:09.057185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1042/2081] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:09 2022] PUT /identity/v3/projects/c5a29b3102eb43fd94bdd568781904a9/users/96449207f1a34ef9b5c433e147de0c5f/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:09.067990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872dff5-0d1f-4d4a-b58e-3113ecf3629b None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:09.068278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872dff5-0d1f-4d4a-b58e-3113ecf3629b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:09.068551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872dff5-0d1f-4d4a-b58e-3113ecf3629b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:09.099773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8872dff5-0d1f-4d4a-b58e-3113ecf3629b None None] MFA Rules not processed for user `96449207f1a34ef9b5c433e147de0c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:09.101285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8872dff5-0d1f-4d4a-b58e-3113ecf3629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:09.180455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1040/2082] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:09.192734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:09.194690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:09.280590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f52ca0-b034-420c-8c40-34c1221ebec1 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:09.284656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b9188-c2ca-455d-b8e1-4aa44e123483 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33183e28de8f487f82f92f0151e7c73c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:09.285278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b9188-c2ca-455d-b8e1-4aa44e123483 service glance] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:09.285594 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b9188-c2ca-455d-b8e1-4aa44e123483 service glance] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:09.285867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b9188-c2ca-455d-b8e1-4aa44e123483 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:09.294216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b9188-c2ca-455d-b8e1-4aa44e123483 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:09.295648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b9188-c2ca-455d-b8e1-4aa44e123483 service glance] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:09.304385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1043/2083] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:11.247858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872dff5-0d1f-4d4a-b58e-3113ecf3629b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:11.249486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8872dff5-0d1f-4d4a-b58e-3113ecf3629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:11.337204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:11.351787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872dff5-0d1f-4d4a-b58e-3113ecf3629b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:11.357892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:11.358952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:11.359383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:11.359799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] PATH_INFO: `/v3/users/0755af23dada46629a2e45240463bce4` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:11.361897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0755af23dada46629a2e45240463bce4)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:11.364119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:11.434285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0755af23dada46629a2e45240463bce4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:11.445457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0755af23dada46629a2e45240463bce4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:11.452324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0755af23dada46629a2e45240463bce4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:11.453267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0755af23dada46629a2e45240463bce4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:11.467405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0755af23dada46629a2e45240463bce4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:11.475502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1041/2084] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:22:11 2022] DELETE /identity/v3/users/0755af23dada46629a2e45240463bce4 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:11.480088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d4c4-5430-426f-b5bf-7bc1c7849e60 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:11.480600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d4c4-5430-426f-b5bf-7bc1c7849e60 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:11.480692 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d4c4-5430-426f-b5bf-7bc1c7849e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:11.500700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-55e9d4c4-5430-426f-b5bf-7bc1c7849e60 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:11.502691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-55e9d4c4-5430-426f-b5bf-7bc1c7849e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:11.569640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1044/2085] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:11.579990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:11.582517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:11.640162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b7bf4-e59b-4285-8231-234b07752db9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:11.644598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08846053-0035-4fb0-85e6-3246bd495005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:11.645209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08846053-0035-4fb0-85e6-3246bd495005 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:11.645511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08846053-0035-4fb0-85e6-3246bd495005 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:11.645782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08846053-0035-4fb0-85e6-3246bd495005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:11.655189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08846053-0035-4fb0-85e6-3246bd495005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:11.656599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08846053-0035-4fb0-85e6-3246bd495005 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:11.666525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1042/2086] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:22:11.985830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9d4c4-5430-426f-b5bf-7bc1c7849e60 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:11.987478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-55e9d4c4-5430-426f-b5bf-7bc1c7849e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:12.000695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9d4c4-5430-426f-b5bf-7bc1c7849e60 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:12.008720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:12.009392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:12.009626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:12.009905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] PATH_INFO: `/v3/projects/e9f84ad3b15a414c9520b95806374a5e` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:12.011127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f84ad3b15a414c9520b95806374a5e)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:12.013081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:12.081080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f84ad3b15a414c9520b95806374a5e', 'request_id': 'req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.088096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f84ad3b15a414c9520b95806374a5e', 'request_id': 'req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.096302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f84ad3b15a414c9520b95806374a5e', 'request_id': 'req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.108581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] The token cache is being invalidate because project e9f84ad3b15a414c9520b95806374a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:22:12.108869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.109588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.111563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1045/2087] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:22:11 2022] DELETE /identity/v3/projects/e9f84ad3b15a414c9520b95806374a5e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:12.115515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08846053-0035-4fb0-85e6-3246bd495005 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:12.125437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08846053-0035-4fb0-85e6-3246bd495005 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:12.184262 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:12.184902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:12.185348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:12.185874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] PATH_INFO: `/v3/users/2200d1c0ef21454a89f2bd377b36f7ac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:12.187034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2200d1c0ef21454a89f2bd377b36f7ac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:12.190007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:12.212960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:12.242274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2200d1c0ef21454a89f2bd377b36f7ac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.249228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2200d1c0ef21454a89f2bd377b36f7ac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.256812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2200d1c0ef21454a89f2bd377b36f7ac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.257630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2200d1c0ef21454a89f2bd377b36f7ac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.269802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2200d1c0ef21454a89f2bd377b36f7ac'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.275653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1043/2088] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:22:12 2022] DELETE /identity/v3/users/2200d1c0ef21454a89f2bd377b36f7ac => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:12.600876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:12.603023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:12.631057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:12.673448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bd49a-ff51-4c5b-9cd6-d6fc624f6841 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:12.775084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:12.776189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:12.777335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:12.777796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] PATH_INFO: `/v3/projects/543e3c6bbad64df68dac4cc6d125956b` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:12.779137 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=543e3c6bbad64df68dac4cc6d125956b)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:12.781914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:12.842684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543e3c6bbad64df68dac4cc6d125956b', 'request_id': 'req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.850963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543e3c6bbad64df68dac4cc6d125956b', 'request_id': 'req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.857203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543e3c6bbad64df68dac4cc6d125956b', 'request_id': 'req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.869591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] The token cache is being invalidate because project 543e3c6bbad64df68dac4cc6d125956b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:22:12.869886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.870543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:12.872303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1046/2089] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:22:12 2022] DELETE /identity/v3/projects/543e3c6bbad64df68dac4cc6d125956b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:12.876477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:12.878139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:12.891701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25210d-807b-482f-a079-c5983a37eeb1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:12.954954 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:12.955793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:12.956278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:12.956572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] PATH_INFO: `/v3/users/888b76a3f851401b8dc0bb89c925a693` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:12.957935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=888b76a3f851401b8dc0bb89c925a693)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:12.959763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:13.014775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888b76a3f851401b8dc0bb89c925a693'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.023383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888b76a3f851401b8dc0bb89c925a693'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.034569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888b76a3f851401b8dc0bb89c925a693'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.035332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888b76a3f851401b8dc0bb89c925a693'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.049482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-111a1165-1da2-4ade-a356-ea38fc41fae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888b76a3f851401b8dc0bb89c925a693'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.055791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1044/2090] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:22:12 2022] DELETE /identity/v3/users/888b76a3f851401b8dc0bb89c925a693 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:13.469462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:13.471553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:13.491522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e2d14-a96b-4862-b03c-aaa7bb86be4e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:13.563150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:13.564316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:13.564600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:13.564868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] PATH_INFO: `/v3/projects/f51104d97af74faf86a72526c3997e0a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:13.566029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f51104d97af74faf86a72526c3997e0a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:13.567934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:13.604931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:13.616287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f51104d97af74faf86a72526c3997e0a', 'request_id': 'req-d39b250e-2ae2-459d-9246-d80a74ecc7f4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.624402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f51104d97af74faf86a72526c3997e0a', 'request_id': 'req-d39b250e-2ae2-459d-9246-d80a74ecc7f4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.630644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f51104d97af74faf86a72526c3997e0a', 'request_id': 'req-d39b250e-2ae2-459d-9246-d80a74ecc7f4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.643076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] The token cache is being invalidate because project f51104d97af74faf86a72526c3997e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:22:13.643363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.644041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d39b250e-2ae2-459d-9246-d80a74ecc7f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:13.645586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1047/2091] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:22:13 2022] DELETE /identity/v3/projects/f51104d97af74faf86a72526c3997e0a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:13.650594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a8cc-b2f7-4a85-8c1b-351c2ce48c36 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:13.651089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a8cc-b2f7-4a85-8c1b-351c2ce48c36 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:13.651308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a8cc-b2f7-4a85-8c1b-351c2ce48c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:13.671879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-fb26a8cc-b2f7-4a85-8c1b-351c2ce48c36 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:13.673231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fb26a8cc-b2f7-4a85-8c1b-351c2ce48c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:13.686052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:13.739112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1045/2092] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:13.747087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29084-85f8-4092-9798-16a02554e75a None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:13.747372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29084-85f8-4092-9798-16a02554e75a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:13.747769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29084-85f8-4092-9798-16a02554e75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:13.770396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0bc29084-85f8-4092-9798-16a02554e75a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:13.771640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0bc29084-85f8-4092-9798-16a02554e75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:13.779600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1048/2093] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:13.785616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26a8cc-b2f7-4a85-8c1b-351c2ce48c36 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:13.796933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26a8cc-b2f7-4a85-8c1b-351c2ce48c36 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:13.803153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:13.803649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:13.803930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:13.804198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:13.804898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:22:13.805198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:13.806916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:13.820264 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1046/2094] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:22:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:22:13.825124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc29084-85f8-4092-9798-16a02554e75a None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:13.842776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc29084-85f8-4092-9798-16a02554e75a None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:13.852112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d36c0-06de-48c1-89ab-437fa30b3f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:13.853726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d36c0-06de-48c1-89ab-437fa30b3f08 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:13.855416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d36c0-06de-48c1-89ab-437fa30b3f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:13.855875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d36c0-06de-48c1-89ab-437fa30b3f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:13.856638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d36c0-06de-48c1-89ab-437fa30b3f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:13.859112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d36c0-06de-48c1-89ab-437fa30b3f08 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:13.905031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1049/2095] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:22:13 2022] POST /identity/v3/projects => generated 370 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:13.906600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:13.918716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd42ae-0e02-4b1f-81e6-d57d84dd0f16 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:13.986690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70600a48-2580-42ad-8ee3-bd680a22e8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:13.987224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70600a48-2580-42ad-8ee3-bd680a22e8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:13.987500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70600a48-2580-42ad-8ee3-bd680a22e8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:13.987771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70600a48-2580-42ad-8ee3-bd680a22e8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:13.988425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70600a48-2580-42ad-8ee3-bd680a22e8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:13.990365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70600a48-2580-42ad-8ee3-bd680a22e8c4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:14.017937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1047/2096] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:22:13 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:14.019570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d36c0-06de-48c1-89ab-437fa30b3f08 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:14.036947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d36c0-06de-48c1-89ab-437fa30b3f08 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:14.042621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450025f-a21d-4391-a812-b19765478b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:14.043254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450025f-a21d-4391-a812-b19765478b5f admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:14.043539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450025f-a21d-4391-a812-b19765478b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:14.043807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450025f-a21d-4391-a812-b19765478b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:14.044518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450025f-a21d-4391-a812-b19765478b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:14.046170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450025f-a21d-4391-a812-b19765478b5f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:14.059956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1050/2097] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:14 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:14.061248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70600a48-2580-42ad-8ee3-bd680a22e8c4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:14.073998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70600a48-2580-42ad-8ee3-bd680a22e8c4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:14.083773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443ef6-e772-4f04-92b3-35d280034e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:14.084921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443ef6-e772-4f04-92b3-35d280034e77 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:14.085272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443ef6-e772-4f04-92b3-35d280034e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:14.085597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443ef6-e772-4f04-92b3-35d280034e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:14.087255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443ef6-e772-4f04-92b3-35d280034e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:14.089068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443ef6-e772-4f04-92b3-35d280034e77 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:14.113182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1048/2098] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:14 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:14.114598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450025f-a21d-4391-a812-b19765478b5f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:14.131632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450025f-a21d-4391-a812-b19765478b5f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:14.137805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:14.138657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:14.139082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:14.139240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] PATH_INFO: `/v3/projects/64564daee3084f369f093513059033eb/users/3c3b49fd331c4e3f90a8080e8e47aefd/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:14.159941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64564daee3084f369f093513059033eb, user_id=3c3b49fd331c4e3f90a8080e8e47aefd, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:14.162681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:14.184207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1051/2099] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:14 2022] PUT /identity/v3/projects/64564daee3084f369f093513059033eb/users/3c3b49fd331c4e3f90a8080e8e47aefd/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:14.195681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e28d3-c1cd-4983-a750-bb9240eb5115 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:14.195963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e28d3-c1cd-4983-a750-bb9240eb5115 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:14.196263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e28d3-c1cd-4983-a750-bb9240eb5115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:14.227543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f37e28d3-c1cd-4983-a750-bb9240eb5115 None None] MFA Rules not processed for user `3c3b49fd331c4e3f90a8080e8e47aefd`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:14.228768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f37e28d3-c1cd-4983-a750-bb9240eb5115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:14.313136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1049/2100] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:22:14.322563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:14.324320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:14.394521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9572a70-0867-405d-8b1c-9e4f92e5545d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:14.399067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33183e28de8f487f82f92f0151e7c73c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:14.399831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:14.400212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:14.400595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:14.410662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:14.412566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:14.423513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1052/2101] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:16.224213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cc806-9fb4-47eb-8463-12527978da42 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:16.224213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cc806-9fb4-47eb-8463-12527978da42 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:16.225252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cc806-9fb4-47eb-8463-12527978da42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:16.244101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4a4cc806-9fb4-47eb-8463-12527978da42 None None] MFA Rules not processed for user `9542521c42b34769a236eb6f268c5358`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:16.245301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4a4cc806-9fb4-47eb-8463-12527978da42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:16.254953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:16.304402 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1050/2102] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:16.854234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:16.855880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:16.930847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d72a-ad8a-4e6c-a9f9-00022295eeec service glance] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:16.935783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:16.936441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:16.936725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:16.937002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] PATH_INFO: `/v3/users/2ededf6b87e34578bc0991e7a560d4ad` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:16.938743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ededf6b87e34578bc0991e7a560d4ad)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:16.940722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:16.943702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:16.996995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ededf6b87e34578bc0991e7a560d4ad'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.005791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ededf6b87e34578bc0991e7a560d4ad'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.007189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ededf6b87e34578bc0991e7a560d4ad'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.022766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ededf6b87e34578bc0991e7a560d4ad'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.023493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ededf6b87e34578bc0991e7a560d4ad'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.035504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1053/2103] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:22:16 2022] DELETE /identity/v3/users/2ededf6b87e34578bc0991e7a560d4ad => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:17.039868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40d8e2-3b68-4817-bde4-e61893cee35e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:17.040141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40d8e2-3b68-4817-bde4-e61893cee35e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:17.040411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40d8e2-3b68-4817-bde4-e61893cee35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:17.059400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8b40d8e2-3b68-4817-bde4-e61893cee35e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:17.060587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8b40d8e2-3b68-4817-bde4-e61893cee35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:17.157621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1051/2104] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:17.167759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:17.169010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:17.229055 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6175-1e04-4a65-88d6-6962d3820cac admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:17.232810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982df3d-3b6f-4c12-acfe-dc79f1c4915d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:17.233448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982df3d-3b6f-4c12-acfe-dc79f1c4915d service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:17.233730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982df3d-3b6f-4c12-acfe-dc79f1c4915d service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:17.234021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982df3d-3b6f-4c12-acfe-dc79f1c4915d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:17.245675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982df3d-3b6f-4c12-acfe-dc79f1c4915d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:17.247469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982df3d-3b6f-4c12-acfe-dc79f1c4915d service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:17.260284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1054/2105] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:17.560079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40d8e2-3b68-4817-bde4-e61893cee35e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:17.561713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8b40d8e2-3b68-4817-bde4-e61893cee35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:17.574048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40d8e2-3b68-4817-bde4-e61893cee35e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:17.578761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:17.579498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:17.579846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:17.580189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] PATH_INFO: `/v3/projects/1cd0f39c1d50454e97caf9625760ab4a` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:17.581449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd0f39c1d50454e97caf9625760ab4a)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:17.583459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:17.634579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd0f39c1d50454e97caf9625760ab4a', 'request_id': 'req-61917713-b734-497a-82a5-42a46bf0e917'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.640062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd0f39c1d50454e97caf9625760ab4a', 'request_id': 'req-61917713-b734-497a-82a5-42a46bf0e917'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.647715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd0f39c1d50454e97caf9625760ab4a', 'request_id': 'req-61917713-b734-497a-82a5-42a46bf0e917'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.660090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] The token cache is being invalidate because project 1cd0f39c1d50454e97caf9625760ab4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:22:17.660374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.661026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-61917713-b734-497a-82a5-42a46bf0e917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:17.662582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1052/2106] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:22:17 2022] DELETE /identity/v3/projects/1cd0f39c1d50454e97caf9625760ab4a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:17.667548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212a4d-86f3-4c37-b9e5-0fa68219a0f0 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:17.667833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212a4d-86f3-4c37-b9e5-0fa68219a0f0 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:17.668108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212a4d-86f3-4c37-b9e5-0fa68219a0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:17.687813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9f212a4d-86f3-4c37-b9e5-0fa68219a0f0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:17.689218 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f212a4d-86f3-4c37-b9e5-0fa68219a0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:17.750390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1055/2107] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:22:17.759578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f5b34-f0bf-447f-85ff-4d872e4fef22 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:17.759868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f5b34-f0bf-447f-85ff-4d872e4fef22 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:17.760143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f5b34-f0bf-447f-85ff-4d872e4fef22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:17.781320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-793f5b34-f0bf-447f-85ff-4d872e4fef22 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:17.783480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-793f5b34-f0bf-447f-85ff-4d872e4fef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:17.796098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1053/2108] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:17.797858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f212a4d-86f3-4c37-b9e5-0fa68219a0f0 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:17.807875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f212a4d-86f3-4c37-b9e5-0fa68219a0f0 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:17.812413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:17.813076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:17.813459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:17.813746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:17.814443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:22:17.814745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:17.819118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:17.831537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1056/2109] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:22:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:22:17.835934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f5b34-f0bf-447f-85ff-4d872e4fef22 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:17.846451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f5b34-f0bf-447f-85ff-4d872e4fef22 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:17.851243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63183eba-7dcd-4c0a-b712-1dc247e67727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:17.851993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63183eba-7dcd-4c0a-b712-1dc247e67727 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:17.852260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63183eba-7dcd-4c0a-b712-1dc247e67727 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:17.852584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63183eba-7dcd-4c0a-b712-1dc247e67727 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:17.853259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63183eba-7dcd-4c0a-b712-1dc247e67727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:17.855217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63183eba-7dcd-4c0a-b712-1dc247e67727 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:17.891638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1054/2110] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:22:17 2022] POST /identity/v3/projects => generated 380 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:22:17.895943 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:17.906217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269809ef-f95b-4b00-a8b6-3cf43a30f877 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:17.915178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:17.966463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc60543-2ac8-4993-a6fc-0321a9d3ce8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:17.967117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc60543-2ac8-4993-a6fc-0321a9d3ce8d admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:17.967417 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc60543-2ac8-4993-a6fc-0321a9d3ce8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:17.967678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc60543-2ac8-4993-a6fc-0321a9d3ce8d admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:17.968347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc60543-2ac8-4993-a6fc-0321a9d3ce8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:17.970995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc60543-2ac8-4993-a6fc-0321a9d3ce8d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:17.993303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1057/2111] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:22:17 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:17.997841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63183eba-7dcd-4c0a-b712-1dc247e67727 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.008650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63183eba-7dcd-4c0a-b712-1dc247e67727 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.013297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42c48c-4842-49d6-a5f7-2a197aa89878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.013932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42c48c-4842-49d6-a5f7-2a197aa89878 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.014213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42c48c-4842-49d6-a5f7-2a197aa89878 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.014485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42c48c-4842-49d6-a5f7-2a197aa89878 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.015172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42c48c-4842-49d6-a5f7-2a197aa89878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.017304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42c48c-4842-49d6-a5f7-2a197aa89878 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.027209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1055/2112] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:17 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:22:18.031097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc60543-2ac8-4993-a6fc-0321a9d3ce8d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.043024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc60543-2ac8-4993-a6fc-0321a9d3ce8d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.047820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cfda9-7866-4115-86ae-caabc46c55fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.048454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cfda9-7866-4115-86ae-caabc46c55fd admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.048737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cfda9-7866-4115-86ae-caabc46c55fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.049016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cfda9-7866-4115-86ae-caabc46c55fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.049754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cfda9-7866-4115-86ae-caabc46c55fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.051380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cfda9-7866-4115-86ae-caabc46c55fd admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.062682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1058/2113] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:18 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:18.066961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42c48c-4842-49d6-a5f7-2a197aa89878 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.072144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:18.078404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42c48c-4842-49d6-a5f7-2a197aa89878 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.084595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669c0e7-6c7f-475e-b2bf-0da6e1837f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.085309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669c0e7-6c7f-475e-b2bf-0da6e1837f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.085642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669c0e7-6c7f-475e-b2bf-0da6e1837f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.085915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669c0e7-6c7f-475e-b2bf-0da6e1837f60 admin admin] PATH_INFO: `/v3/projects/cb751f8fc96c4822815af739a779af30/users/ea7edb05d378471dad17456d6f3fa052/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.102916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669c0e7-6c7f-475e-b2bf-0da6e1837f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb751f8fc96c4822815af739a779af30, user_id=ea7edb05d378471dad17456d6f3fa052, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.105702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669c0e7-6c7f-475e-b2bf-0da6e1837f60 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.120266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1056/2114] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:18 2022] PUT /identity/v3/projects/cb751f8fc96c4822815af739a779af30/users/ea7edb05d378471dad17456d6f3fa052/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:18.131534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69997-9c41-4697-8fac-be9e752ab575 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.131818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69997-9c41-4697-8fac-be9e752ab575 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.132112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69997-9c41-4697-8fac-be9e752ab575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.165694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2de69997-9c41-4697-8fac-be9e752ab575 None None] MFA Rules not processed for user `ea7edb05d378471dad17456d6f3fa052`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:18.166942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2de69997-9c41-4697-8fac-be9e752ab575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:18.233970 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1059/2115] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:22:18.238468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669c0e7-6c7f-475e-b2bf-0da6e1837f60 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.319466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669c0e7-6c7f-475e-b2bf-0da6e1837f60 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.324276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a65a23-8dfc-4437-a495-14ff43d941e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.324884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a65a23-8dfc-4437-a495-14ff43d941e4 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.325169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a65a23-8dfc-4437-a495-14ff43d941e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.325462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a65a23-8dfc-4437-a495-14ff43d941e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.326141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a65a23-8dfc-4437-a495-14ff43d941e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.328049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a65a23-8dfc-4437-a495-14ff43d941e4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.355401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1057/2116] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:22:18 2022] POST /identity/v3/projects => generated 380 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:22:18.359453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de69997-9c41-4697-8fac-be9e752ab575 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.370621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de69997-9c41-4697-8fac-be9e752ab575 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.434435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797e76-79ae-4659-9aca-751ea85bc418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.435058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797e76-79ae-4659-9aca-751ea85bc418 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.435337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797e76-79ae-4659-9aca-751ea85bc418 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.435611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797e76-79ae-4659-9aca-751ea85bc418 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.436284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da797e76-79ae-4659-9aca-751ea85bc418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.439318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da797e76-79ae-4659-9aca-751ea85bc418 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.461514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1060/2117] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:22:18 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:18.465535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a65a23-8dfc-4437-a495-14ff43d941e4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.476137 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a65a23-8dfc-4437-a495-14ff43d941e4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.480961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c662b-d852-452f-915c-bb9737e41d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.481621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c662b-d852-452f-915c-bb9737e41d2f admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.481902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c662b-d852-452f-915c-bb9737e41d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.482192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c662b-d852-452f-915c-bb9737e41d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.482875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c662b-d852-452f-915c-bb9737e41d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.484495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c662b-d852-452f-915c-bb9737e41d2f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.496874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1058/2118] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:18 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:18.501916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797e76-79ae-4659-9aca-751ea85bc418 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.513052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797e76-79ae-4659-9aca-751ea85bc418 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.518752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfcf69-ab4a-413b-9737-f3a801d22743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.519439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfcf69-ab4a-413b-9737-f3a801d22743 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.519860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfcf69-ab4a-413b-9737-f3a801d22743 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.520146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfcf69-ab4a-413b-9737-f3a801d22743 admin admin] PATH_INFO: `/v3/projects/785fdff2babf40bf83bfef792a223f12/users/a21ea2c51f6942b59d168c2c4661eeb8/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.530898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:18.539385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfcf69-ab4a-413b-9737-f3a801d22743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=785fdff2babf40bf83bfef792a223f12, user_id=a21ea2c51f6942b59d168c2c4661eeb8, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.542178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfcf69-ab4a-413b-9737-f3a801d22743 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.555117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1061/2119] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:18 2022] PUT /identity/v3/projects/785fdff2babf40bf83bfef792a223f12/users/a21ea2c51f6942b59d168c2c4661eeb8/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:22:18.559245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c662b-d852-452f-915c-bb9737e41d2f admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.570636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c662b-d852-452f-915c-bb9737e41d2f admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.636311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0571694-b84c-4d9e-83d7-fe2227b4a1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.636960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0571694-b84c-4d9e-83d7-fe2227b4a1ea admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.637323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0571694-b84c-4d9e-83d7-fe2227b4a1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.637647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0571694-b84c-4d9e-83d7-fe2227b4a1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.638352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0571694-b84c-4d9e-83d7-fe2227b4a1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.639968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0571694-b84c-4d9e-83d7-fe2227b4a1ea admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.651990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1059/2120] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:18 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:18.655518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfcf69-ab4a-413b-9737-f3a801d22743 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.666225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfcf69-ab4a-413b-9737-f3a801d22743 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.670697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.671353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.671630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.671900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] PATH_INFO: `/v3/projects/785fdff2babf40bf83bfef792a223f12/users/a21ea2c51f6942b59d168c2c4661eeb8/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.674393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=785fdff2babf40bf83bfef792a223f12, user_id=a21ea2c51f6942b59d168c2c4661eeb8, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.676998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.686928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1062/2121] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:18 2022] PUT /identity/v3/projects/785fdff2babf40bf83bfef792a223f12/users/a21ea2c51f6942b59d168c2c4661eeb8/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:18.698815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028401-0cbc-4153-b441-de0c82cea453 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.699108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028401-0cbc-4153-b441-de0c82cea453 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.699763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028401-0cbc-4153-b441-de0c82cea453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.727680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-0f028401-0cbc-4153-b441-de0c82cea453 None None] MFA Rules not processed for user `a21ea2c51f6942b59d168c2c4661eeb8`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:18.729221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0f028401-0cbc-4153-b441-de0c82cea453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:18.770791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:18.802376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1060/2122] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:18 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:18.814517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.816328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:18.876446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a15a3-daa8-44ef-8fb5-c9d0e4a78de1 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:18.880175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5bd78-9109-4249-b4e0-70d594166db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:18.881034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5bd78-9109-4249-b4e0-70d594166db0 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:18.881208 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5bd78-9109-4249-b4e0-70d594166db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:18.881457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5bd78-9109-4249-b4e0-70d594166db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:18.894558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5bd78-9109-4249-b4e0-70d594166db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:18.896002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5bd78-9109-4249-b4e0-70d594166db0 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:18.905737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1063/2123] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:18 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:18.984689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028401-0cbc-4153-b441-de0c82cea453 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:18.997213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028401-0cbc-4153-b441-de0c82cea453 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:19.003778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6c446-eaa1-45f0-a03d-89d0724e92de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:19.004443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c446-eaa1-45f0-a03d-89d0724e92de service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:19.004722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c446-eaa1-45f0-a03d-89d0724e92de service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:19.005009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c446-eaa1-45f0-a03d-89d0724e92de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:19.067938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c446-eaa1-45f0-a03d-89d0724e92de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:19.069461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c446-eaa1-45f0-a03d-89d0724e92de service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:19.078801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1061/2124] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:28.901306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=37,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:29.074944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=40 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:36.510982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5bd78-9109-4249-b4e0-70d594166db0 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:36.588610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5bd78-9109-4249-b4e0-70d594166db0 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:36.593295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:36.593971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:36.594249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:36.594525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] PATH_INFO: `/v3/users/ea7edb05d378471dad17456d6f3fa052` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:36.595700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7edb05d378471dad17456d6f3fa052)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:36.597725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:36.645229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7edb05d378471dad17456d6f3fa052'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:36.652732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7edb05d378471dad17456d6f3fa052'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:36.653343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7edb05d378471dad17456d6f3fa052'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:36.665909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7edb05d378471dad17456d6f3fa052'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:36.666581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7edb05d378471dad17456d6f3fa052'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:36.680678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1064/2125] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:22:36 2022] DELETE /identity/v3/users/ea7edb05d378471dad17456d6f3fa052 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:36.685312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0a9d1-166f-4791-8e63-0c213aaed3c1 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:36.685873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0a9d1-166f-4791-8e63-0c213aaed3c1 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:36.685873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0a9d1-166f-4791-8e63-0c213aaed3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:36.705740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f0d0a9d1-166f-4791-8e63-0c213aaed3c1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:36.706919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f0d0a9d1-166f-4791-8e63-0c213aaed3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:36.765878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1062/2126] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:36.775082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:36.776497 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:36.836674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7a5b0-4931-40b1-8e78-17e3eeebcb8f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:36.840407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a42ea-5c86-4dd1-8b65-06f3230d6939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:36.841036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a42ea-5c86-4dd1-8b65-06f3230d6939 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:36.841311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a42ea-5c86-4dd1-8b65-06f3230d6939 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:36.841608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a42ea-5c86-4dd1-8b65-06f3230d6939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:36.852534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a42ea-5c86-4dd1-8b65-06f3230d6939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:36.854030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a42ea-5c86-4dd1-8b65-06f3230d6939 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:36.864223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1065/2127] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:37.239728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0a9d1-166f-4791-8e63-0c213aaed3c1 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:37.241185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f0d0a9d1-166f-4791-8e63-0c213aaed3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:37.253940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0a9d1-166f-4791-8e63-0c213aaed3c1 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:37.258926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:37.259901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:37.268501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:37.268956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] PATH_INFO: `/v3/projects/cb751f8fc96c4822815af739a779af30` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:37.270460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb751f8fc96c4822815af739a779af30)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:37.272635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:37.329694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb751f8fc96c4822815af739a779af30', 'request_id': 'req-2a630e24-1cd9-4215-bc71-e7c81d24bed9'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.335276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb751f8fc96c4822815af739a779af30', 'request_id': 'req-2a630e24-1cd9-4215-bc71-e7c81d24bed9'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.346419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb751f8fc96c4822815af739a779af30', 'request_id': 'req-2a630e24-1cd9-4215-bc71-e7c81d24bed9'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.358699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] The token cache is being invalidate because project cb751f8fc96c4822815af739a779af30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:22:37.358988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.359659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.361179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1063/2128] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:22:37 2022] DELETE /identity/v3/projects/cb751f8fc96c4822815af739a779af30 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:37.365329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a42ea-5c86-4dd1-8b65-06f3230d6939 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:37.375706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a42ea-5c86-4dd1-8b65-06f3230d6939 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:37.395507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:37.435651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:37.436288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:37.436567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:37.437324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] PATH_INFO: `/v3/users/a21ea2c51f6942b59d168c2c4661eeb8` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:37.438521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21ea2c51f6942b59d168c2c4661eeb8)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:37.440457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:37.488028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21ea2c51f6942b59d168c2c4661eeb8'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.496001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21ea2c51f6942b59d168c2c4661eeb8'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.496637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21ea2c51f6942b59d168c2c4661eeb8'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.509051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21ea2c51f6942b59d168c2c4661eeb8'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.509713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-796f66a4-ba54-4ae6-8e5e-aca338cb4a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21ea2c51f6942b59d168c2c4661eeb8'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.523725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1066/2129] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:22:37 2022] DELETE /identity/v3/users/a21ea2c51f6942b59d168c2c4661eeb8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:22:37.808363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:37.809918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:37.827012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a630e24-1cd9-4215-bc71-e7c81d24bed9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:37.889315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:37.890590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:37.890960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:37.891242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] PATH_INFO: `/v3/projects/785fdff2babf40bf83bfef792a223f12` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:37.892421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=785fdff2babf40bf83bfef792a223f12)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:37.894408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:37.923523 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:37.941522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785fdff2babf40bf83bfef792a223f12', 'request_id': 'req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.946968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785fdff2babf40bf83bfef792a223f12', 'request_id': 'req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.954745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785fdff2babf40bf83bfef792a223f12', 'request_id': 'req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.965871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] The token cache is being invalidate because project 785fdff2babf40bf83bfef792a223f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:22:37.966158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.966798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8d176bb-526d-4b83-9af2-d6955d3b3ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:22:37.968449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1064/2130] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:22:37 2022] DELETE /identity/v3/projects/785fdff2babf40bf83bfef792a223f12 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:37.973331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a7bea-f9b2-4b3c-9573-bbef60eab1a3 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:37.973681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a7bea-f9b2-4b3c-9573-bbef60eab1a3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:37.974046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a7bea-f9b2-4b3c-9573-bbef60eab1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:37.995400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a43a7bea-f9b2-4b3c-9573-bbef60eab1a3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:37.996632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a43a7bea-f9b2-4b3c-9573-bbef60eab1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:38.055911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1067/2131] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:38.064266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085c4c4-b48e-414a-b960-4cdda012db33 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.064550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085c4c4-b48e-414a-b960-4cdda012db33 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.064822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085c4c4-b48e-414a-b960-4cdda012db33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.083944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6085c4c4-b48e-414a-b960-4cdda012db33 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:38.085128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6085c4c4-b48e-414a-b960-4cdda012db33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:38.093183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1065/2132] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:38.097221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a7bea-f9b2-4b3c-9573-bbef60eab1a3 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.107191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a7bea-f9b2-4b3c-9573-bbef60eab1a3 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.112567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.113192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.113500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.113778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.114481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:22:38.114810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.116420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.129232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1068/2133] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:22:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:22:38.133470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085c4c4-b48e-414a-b960-4cdda012db33 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.142945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085c4c4-b48e-414a-b960-4cdda012db33 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.148108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148e467-5aff-4608-934c-a4d833f7b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.148757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e467-5aff-4608-934c-a4d833f7b59b admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.149076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e467-5aff-4608-934c-a4d833f7b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.149328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e467-5aff-4608-934c-a4d833f7b59b admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.150118 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148e467-5aff-4608-934c-a4d833f7b59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.154346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148e467-5aff-4608-934c-a4d833f7b59b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.183726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1066/2134] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:22:38 2022] POST /identity/v3/projects => generated 378 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:22:38.188747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.199366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298fa9e-bc85-4368-89f2-4669959218f2 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.236370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:38.271608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c56b-15a8-431a-a6f2-3e80e3b29f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.272247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c56b-15a8-431a-a6f2-3e80e3b29f90 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.272586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c56b-15a8-431a-a6f2-3e80e3b29f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.272869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c56b-15a8-431a-a6f2-3e80e3b29f90 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.273684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2c56b-15a8-431a-a6f2-3e80e3b29f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.275875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2c56b-15a8-431a-a6f2-3e80e3b29f90 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.303494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1069/2135] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:22:38 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:22:38.304714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148e467-5aff-4608-934c-a4d833f7b59b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.317830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148e467-5aff-4608-934c-a4d833f7b59b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.323658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46a7e1-4f71-41fe-a3b1-6d7476cbbd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.324365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46a7e1-4f71-41fe-a3b1-6d7476cbbd1d admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.324655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46a7e1-4f71-41fe-a3b1-6d7476cbbd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.324930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46a7e1-4f71-41fe-a3b1-6d7476cbbd1d admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.325818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46a7e1-4f71-41fe-a3b1-6d7476cbbd1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.327573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46a7e1-4f71-41fe-a3b1-6d7476cbbd1d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.342154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1067/2136] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:38 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:38.343333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c56b-15a8-431a-a6f2-3e80e3b29f90 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.354277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c56b-15a8-431a-a6f2-3e80e3b29f90 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.362311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e6849-252a-436e-aaa0-dc8787209ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.363379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e6849-252a-436e-aaa0-dc8787209ecc admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.363379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e6849-252a-436e-aaa0-dc8787209ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.363565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e6849-252a-436e-aaa0-dc8787209ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.364214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e6849-252a-436e-aaa0-dc8787209ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.365989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e6849-252a-436e-aaa0-dc8787209ecc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.384573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1070/2137] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:38 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:38.385783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46a7e1-4f71-41fe-a3b1-6d7476cbbd1d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.403820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46a7e1-4f71-41fe-a3b1-6d7476cbbd1d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.414699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f6959-301e-4487-ba8b-9196a00d5d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.415981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f6959-301e-4487-ba8b-9196a00d5d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.429696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f6959-301e-4487-ba8b-9196a00d5d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.429696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f6959-301e-4487-ba8b-9196a00d5d34 admin admin] PATH_INFO: `/v3/projects/af7bee3b6f004bfea65263606af8829c/users/8c158945c2d64b798b3f1f72f7e99985/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.440547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f6959-301e-4487-ba8b-9196a00d5d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7bee3b6f004bfea65263606af8829c, user_id=8c158945c2d64b798b3f1f72f7e99985, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.443704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f6959-301e-4487-ba8b-9196a00d5d34 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.460561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1068/2138] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:38 2022] PUT /identity/v3/projects/af7bee3b6f004bfea65263606af8829c/users/8c158945c2d64b798b3f1f72f7e99985/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:22:38.471445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e777c79-0892-438f-9463-fe5224f20d47 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.471746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e777c79-0892-438f-9463-fe5224f20d47 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.472002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e777c79-0892-438f-9463-fe5224f20d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.507027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-5e777c79-0892-438f-9463-fe5224f20d47 None None] MFA Rules not processed for user `8c158945c2d64b798b3f1f72f7e99985`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:38.508617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5e777c79-0892-438f-9463-fe5224f20d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:38.599620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1071/2139] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:38.604155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f6959-301e-4487-ba8b-9196a00d5d34 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.690126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f6959-301e-4487-ba8b-9196a00d5d34 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.699572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703e908-60a2-4884-8c35-10b122d5533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.700223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e908-60a2-4884-8c35-10b122d5533d admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.700501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e908-60a2-4884-8c35-10b122d5533d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.700771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e908-60a2-4884-8c35-10b122d5533d admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.701492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703e908-60a2-4884-8c35-10b122d5533d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.704361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703e908-60a2-4884-8c35-10b122d5533d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.736596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1069/2140] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:22:38 2022] POST /identity/v3/projects => generated 378 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:38.737938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e777c79-0892-438f-9463-fe5224f20d47 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.749808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e777c79-0892-438f-9463-fe5224f20d47 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.806247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6562f8d-9f0e-4fcd-b711-5571f731e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.806945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6562f8d-9f0e-4fcd-b711-5571f731e3ac admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.807226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6562f8d-9f0e-4fcd-b711-5571f731e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.807534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6562f8d-9f0e-4fcd-b711-5571f731e3ac admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.808426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6562f8d-9f0e-4fcd-b711-5571f731e3ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.810590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6562f8d-9f0e-4fcd-b711-5571f731e3ac admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.832328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1072/2141] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:22:38 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:22:38.836562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703e908-60a2-4884-8c35-10b122d5533d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.847158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703e908-60a2-4884-8c35-10b122d5533d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.851902 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8595f4-2050-427c-ba4c-0397fd3c9543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.852513 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8595f4-2050-427c-ba4c-0397fd3c9543 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.852794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8595f4-2050-427c-ba4c-0397fd3c9543 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.853068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8595f4-2050-427c-ba4c-0397fd3c9543 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.853795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8595f4-2050-427c-ba4c-0397fd3c9543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.855400 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8595f4-2050-427c-ba4c-0397fd3c9543 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.866595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1070/2142] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:38 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:38.871128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6562f8d-9f0e-4fcd-b711-5571f731e3ac admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.881932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6562f8d-9f0e-4fcd-b711-5571f731e3ac admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.887270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a390b7-bdeb-4d05-bb19-e60e75c7b747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.887930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a390b7-bdeb-4d05-bb19-e60e75c7b747 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.888211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a390b7-bdeb-4d05-bb19-e60e75c7b747 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.888485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a390b7-bdeb-4d05-bb19-e60e75c7b747 admin admin] PATH_INFO: `/v3/projects/c349269012244a8bbdfa61ac62c89e07/users/9c3c89ab83ed43eb82444dedae9b4c30/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.902990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a390b7-bdeb-4d05-bb19-e60e75c7b747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c349269012244a8bbdfa61ac62c89e07, user_id=9c3c89ab83ed43eb82444dedae9b4c30, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:38.905721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a390b7-bdeb-4d05-bb19-e60e75c7b747 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:38.918245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1073/2143] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:38 2022] PUT /identity/v3/projects/c349269012244a8bbdfa61ac62c89e07/users/9c3c89ab83ed43eb82444dedae9b4c30/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:38.922684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8595f4-2050-427c-ba4c-0397fd3c9543 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:38.932475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8595f4-2050-427c-ba4c-0397fd3c9543 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:38.953544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:38.995623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177249-f40f-4e45-bd84-3c33d5a1345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:38.996385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177249-f40f-4e45-bd84-3c33d5a1345a admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:38.996643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177249-f40f-4e45-bd84-3c33d5a1345a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:38.997022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177249-f40f-4e45-bd84-3c33d5a1345a admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:38.997868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177249-f40f-4e45-bd84-3c33d5a1345a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:39.001083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177249-f40f-4e45-bd84-3c33d5a1345a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:39.011701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1071/2144] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:22:38 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:22:39.016396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a390b7-bdeb-4d05-bb19-e60e75c7b747 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:39.022103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:39.028327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a390b7-bdeb-4d05-bb19-e60e75c7b747 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:39.034025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:39.034690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:39.034973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:39.035248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] PATH_INFO: `/v3/projects/c349269012244a8bbdfa61ac62c89e07/users/9c3c89ab83ed43eb82444dedae9b4c30/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:39.037249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c349269012244a8bbdfa61ac62c89e07, user_id=9c3c89ab83ed43eb82444dedae9b4c30, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:39.040129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:39.050885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1074/2145] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:22:39 2022] PUT /identity/v3/projects/c349269012244a8bbdfa61ac62c89e07/users/9c3c89ab83ed43eb82444dedae9b4c30/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:22:39.062097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d23e8-9add-48ff-8576-284b184dcbb6 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:39.062381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d23e8-9add-48ff-8576-284b184dcbb6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:39.062655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d23e8-9add-48ff-8576-284b184dcbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:39.092012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-793d23e8-9add-48ff-8576-284b184dcbb6 None None] MFA Rules not processed for user `9c3c89ab83ed43eb82444dedae9b4c30`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:22:39.094087 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-793d23e8-9add-48ff-8576-284b184dcbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:39.162533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1072/2146] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:22:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:22:39.173177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:39.174837 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:22:39.239476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523629c-f965-44ce-8056-27274e93876e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:39.243207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:39.243824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:39.244103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:39.244377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:39.303419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:39.304890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:39.314677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1075/2147] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:22:49.159608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=52 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:49.309763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:22:51.222463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d23e8-9add-48ff-8576-284b184dcbb6 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:22:51.232233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d23e8-9add-48ff-8576-284b184dcbb6 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:22:51.235966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c1b37-2e46-4687-81af-2c8942123d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:22:51.236586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c1b37-2e46-4687-81af-2c8942123d04 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:22:51.236867 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c1b37-2e46-4687-81af-2c8942123d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:22:51.237139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c1b37-2e46-4687-81af-2c8942123d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:22:51.246452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c1b37-2e46-4687-81af-2c8942123d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:22:51.247894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c1b37-2e46-4687-81af-2c8942123d04 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:22:51.257827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1073/2148] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:22:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:23:01.253766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=6 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:23:45.946621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:45.948042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:46.018345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0a5ca-e144-47c7-970d-c3c4f80cb2df service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:46.023540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:46.024166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:46.024443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:46.024718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:46.026515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:46.039252 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:46.040824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:46.052011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1076/2149] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:23:45 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:23:52.828048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df377a46-75b6-493b-b838-b732ddf81569 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:52.828558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df377a46-75b6-493b-b838-b732ddf81569 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:52.828675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df377a46-75b6-493b-b838-b732ddf81569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:52.847751 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-df377a46-75b6-493b-b838-b732ddf81569 None None] MFA Rules not processed for user `f3dab07011da408c97ed90d58f2c155b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:23:52.848886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-df377a46-75b6-493b-b838-b732ddf81569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:52.903862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1074/2150] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:23:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:23:56.047489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=32 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:23:56.778530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:56.844416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92248d-ed79-41e0-b3de-80249f4e7803 service placement] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:56.850086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:56.850810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:56.851162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:56.852049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] PATH_INFO: `/v3/users/8c158945c2d64b798b3f1f72f7e99985` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:56.853236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c158945c2d64b798b3f1f72f7e99985)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:56.856628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:56.914687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c158945c2d64b798b3f1f72f7e99985'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:56.923340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c158945c2d64b798b3f1f72f7e99985'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:56.924056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c158945c2d64b798b3f1f72f7e99985'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:56.936511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c158945c2d64b798b3f1f72f7e99985'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:56.937173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c158945c2d64b798b3f1f72f7e99985'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:56.960461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1077/2151] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:23:56 2022] DELETE /identity/v3/users/8c158945c2d64b798b3f1f72f7e99985 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:23:56.962759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1504-cfb3-4069-b622-0bc064ee9b69 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:56.963274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1504-cfb3-4069-b622-0bc064ee9b69 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:56.963369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1504-cfb3-4069-b622-0bc064ee9b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:56.983704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e9cf1504-cfb3-4069-b622-0bc064ee9b69 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:23:56.986332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e9cf1504-cfb3-4069-b622-0bc064ee9b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:57.056033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1075/2152] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:23:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:23:57.065777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:57.067309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:57.138710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee07fb1-91a4-4daa-9d96-bf1bbf041b80 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:57.143300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6168c-0b60-4837-a896-2e19f0c42c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:57.143930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6168c-0b60-4837-a896-2e19f0c42c7b service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:57.144209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6168c-0b60-4837-a896-2e19f0c42c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:57.144479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6168c-0b60-4837-a896-2e19f0c42c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:57.154479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6168c-0b60-4837-a896-2e19f0c42c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:57.156270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6168c-0b60-4837-a896-2e19f0c42c7b service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:57.166774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1078/2153] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:23:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:23:57.466257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf1504-cfb3-4069-b622-0bc064ee9b69 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:57.467772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e9cf1504-cfb3-4069-b622-0bc064ee9b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:57.494555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf1504-cfb3-4069-b622-0bc064ee9b69 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:57.502309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:57.502993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:57.503289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:57.503838 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] PATH_INFO: `/v3/projects/af7bee3b6f004bfea65263606af8829c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:57.504909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7bee3b6f004bfea65263606af8829c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:57.507184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:57.568395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7bee3b6f004bfea65263606af8829c', 'request_id': 'req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.578734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7bee3b6f004bfea65263606af8829c', 'request_id': 'req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.589124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7bee3b6f004bfea65263606af8829c', 'request_id': 'req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.602294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] The token cache is being invalidate because project af7bee3b6f004bfea65263606af8829c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:23:57.602582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.603220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ac74a260-e5f2-4902-9c9c-d3f3bb06c74e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.604734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1076/2154] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:23:57 2022] DELETE /identity/v3/projects/af7bee3b6f004bfea65263606af8829c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:23:57.608876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6168c-0b60-4837-a896-2e19f0c42c7b service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:57.619661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6168c-0b60-4837-a896-2e19f0c42c7b service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:57.639977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:23:57.686652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:57.687340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:57.687734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:57.688615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] PATH_INFO: `/v3/users/9c3c89ab83ed43eb82444dedae9b4c30` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:57.690200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3c89ab83ed43eb82444dedae9b4c30)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:57.692339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:57.749570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3c89ab83ed43eb82444dedae9b4c30'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.758628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3c89ab83ed43eb82444dedae9b4c30'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.759589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3c89ab83ed43eb82444dedae9b4c30'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.775675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3c89ab83ed43eb82444dedae9b4c30'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.776445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3c89ab83ed43eb82444dedae9b4c30'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:57.793148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1079/2155] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:23:57 2022] DELETE /identity/v3/users/9c3c89ab83ed43eb82444dedae9b4c30 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:23:57.868524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920f84-d0e0-4f05-bd24-eff6a180c505 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:57.869259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920f84-d0e0-4f05-bd24-eff6a180c505 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:57.869836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920f84-d0e0-4f05-bd24-eff6a180c505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:57.892477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-aa920f84-d0e0-4f05-bd24-eff6a180c505 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:23:57.894555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-aa920f84-d0e0-4f05-bd24-eff6a180c505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:57.929640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:23:57.994718 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1077/2156] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:23:57 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:23:58.070978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:58.074235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:58.181440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572ccc8-992c-4dcb-8981-034d245fb60b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:58.190922 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0727932-823a-49a5-a891-186b1af8bd93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:58.191886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727932-823a-49a5-a891-186b1af8bd93 service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:58.192485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727932-823a-49a5-a891-186b1af8bd93 service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:58.193057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727932-823a-49a5-a891-186b1af8bd93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:58.206412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0727932-823a-49a5-a891-186b1af8bd93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:58.208843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0727932-823a-49a5-a891-186b1af8bd93 service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:58.224043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1080/2157] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:23:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:23:58.501953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa920f84-d0e0-4f05-bd24-eff6a180c505 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:58.504090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-aa920f84-d0e0-4f05-bd24-eff6a180c505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:58.668047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa920f84-d0e0-4f05-bd24-eff6a180c505 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:58.674952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:58.676145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:58.676499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:58.677116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] PATH_INFO: `/v3/projects/c349269012244a8bbdfa61ac62c89e07` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:58.679373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c349269012244a8bbdfa61ac62c89e07)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:58.681890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:58.790814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c349269012244a8bbdfa61ac62c89e07', 'request_id': 'req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:58.818973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c349269012244a8bbdfa61ac62c89e07', 'request_id': 'req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:58.832372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c349269012244a8bbdfa61ac62c89e07', 'request_id': 'req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:58.846600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] The token cache is being invalidate because project c349269012244a8bbdfa61ac62c89e07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:23:58.846888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:58.847542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c0bdd68e-ff13-47e3-9454-4aa05d62b07b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:58.849054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1078/2158] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:23:58 2022] DELETE /identity/v3/projects/c349269012244a8bbdfa61ac62c89e07 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:23:58.854097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112557b7-f751-4ae5-89a4-7ab5e5dc2486 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:58.854497 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112557b7-f751-4ae5-89a4-7ab5e5dc2486 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:58.854877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112557b7-f751-4ae5-89a4-7ab5e5dc2486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:58.881488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-112557b7-f751-4ae5-89a4-7ab5e5dc2486 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:23:58.882964 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-112557b7-f751-4ae5-89a4-7ab5e5dc2486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:58.913140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:23:58.954250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1081/2159] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:23:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:23:58.963796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414954f2-93ce-4c1a-aa6f-23bc7ebcf22a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:58.964197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414954f2-93ce-4c1a-aa6f-23bc7ebcf22a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:58.964618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414954f2-93ce-4c1a-aa6f-23bc7ebcf22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:58.985227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-414954f2-93ce-4c1a-aa6f-23bc7ebcf22a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:23:58.986622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-414954f2-93ce-4c1a-aa6f-23bc7ebcf22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:58.997269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1079/2160] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:23:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:23:59.001676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112557b7-f751-4ae5-89a4-7ab5e5dc2486 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.012113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112557b7-f751-4ae5-89a4-7ab5e5dc2486 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.017794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.018423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.018708 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.018986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.019677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:23:59.019980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.021908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.037457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1082/2161] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:23:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:23:59.039675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414954f2-93ce-4c1a-aa6f-23bc7ebcf22a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.049653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414954f2-93ce-4c1a-aa6f-23bc7ebcf22a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.054290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e03f7-6dc2-41da-9d65-c65f7baa6d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.054904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e03f7-6dc2-41da-9d65-c65f7baa6d4d admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.055180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e03f7-6dc2-41da-9d65-c65f7baa6d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.055449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e03f7-6dc2-41da-9d65-c65f7baa6d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.056109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e03f7-6dc2-41da-9d65-c65f7baa6d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.058509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e03f7-6dc2-41da-9d65-c65f7baa6d4d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.088810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1080/2162] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:23:59 2022] POST /identity/v3/projects => generated 368 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:23:59.093086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.113906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a714afd-2ba0-4ae0-a175-6f366b2b85f0 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.212156 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e20d0a-c3d2-4a20-afe0-92253d86a8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.212969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e20d0a-c3d2-4a20-afe0-92253d86a8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.213568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e20d0a-c3d2-4a20-afe0-92253d86a8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.213859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e20d0a-c3d2-4a20-afe0-92253d86a8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.214607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e20d0a-c3d2-4a20-afe0-92253d86a8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.216978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e20d0a-c3d2-4a20-afe0-92253d86a8a0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.244445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1083/2163] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:23:59 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:23:59.248617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e03f7-6dc2-41da-9d65-c65f7baa6d4d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.264412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e03f7-6dc2-41da-9d65-c65f7baa6d4d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.273243 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe8b25-c6fc-4d58-99cd-a01a8371e119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.273960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe8b25-c6fc-4d58-99cd-a01a8371e119 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.274560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe8b25-c6fc-4d58-99cd-a01a8371e119 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.275153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe8b25-c6fc-4d58-99cd-a01a8371e119 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.275967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe8b25-c6fc-4d58-99cd-a01a8371e119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.277651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe8b25-c6fc-4d58-99cd-a01a8371e119 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.292787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1081/2164] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:23:59 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:23:59.294326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e20d0a-c3d2-4a20-afe0-92253d86a8a0 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.306619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e20d0a-c3d2-4a20-afe0-92253d86a8a0 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.313436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e78863-d597-484f-80cf-36fe558c9d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.315013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e78863-d597-484f-80cf-36fe558c9d14 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.315013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e78863-d597-484f-80cf-36fe558c9d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.315013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e78863-d597-484f-80cf-36fe558c9d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.315447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e78863-d597-484f-80cf-36fe558c9d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.317066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e78863-d597-484f-80cf-36fe558c9d14 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.332168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1084/2165] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:23:59 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:23:59.336529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe8b25-c6fc-4d58-99cd-a01a8371e119 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.351201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe8b25-c6fc-4d58-99cd-a01a8371e119 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.358546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cfae5-b576-4c73-bdfc-716f7633ca4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.359411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cfae5-b576-4c73-bdfc-716f7633ca4c admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.359744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cfae5-b576-4c73-bdfc-716f7633ca4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.360113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cfae5-b576-4c73-bdfc-716f7633ca4c admin admin] PATH_INFO: `/v3/projects/77f7467a1e4e44eb8996282966c8d5cd/users/209f907510a04bb4950a5a47559de27a/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.382228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cfae5-b576-4c73-bdfc-716f7633ca4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f7467a1e4e44eb8996282966c8d5cd, user_id=209f907510a04bb4950a5a47559de27a, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.390246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cfae5-b576-4c73-bdfc-716f7633ca4c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.409940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:23:59.478801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1082/2166] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:23:59 2022] PUT /identity/v3/projects/77f7467a1e4e44eb8996282966c8d5cd/users/209f907510a04bb4950a5a47559de27a/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:23:59.492897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d78ef-50cf-4729-87b3-e0cdca31b6e2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.493385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d78ef-50cf-4729-87b3-e0cdca31b6e2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.493924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d78ef-50cf-4729-87b3-e0cdca31b6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.543142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6c1d78ef-50cf-4729-87b3-e0cdca31b6e2 None None] MFA Rules not processed for user `209f907510a04bb4950a5a47559de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:23:59.544801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6c1d78ef-50cf-4729-87b3-e0cdca31b6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:59.646116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1085/2167] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:23:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:23:59.652166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cfae5-b576-4c73-bdfc-716f7633ca4c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.705215 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d78ef-50cf-4729-87b3-e0cdca31b6e2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.708299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6c1d78ef-50cf-4729-87b3-e0cdca31b6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:23:59.764191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cfae5-b576-4c73-bdfc-716f7633ca4c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.767991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:23:59.770649 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7d3a4-8a0a-47b3-b26c-126f4cdc76de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.771337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7d3a4-8a0a-47b3-b26c-126f4cdc76de admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.772291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7d3a4-8a0a-47b3-b26c-126f4cdc76de admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.772807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7d3a4-8a0a-47b3-b26c-126f4cdc76de admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.773621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7d3a4-8a0a-47b3-b26c-126f4cdc76de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.776643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7d3a4-8a0a-47b3-b26c-126f4cdc76de admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.820283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d78ef-50cf-4729-87b3-e0cdca31b6e2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.826389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.827566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.828250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.829714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] PATH_INFO: `/v3/users/96449207f1a34ef9b5c433e147de0c5f` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.833799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1083/2168] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:23:59 2022] POST /identity/v3/projects => generated 370 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:23:59.834981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7d3a4-8a0a-47b3-b26c-126f4cdc76de admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.836075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=96449207f1a34ef9b5c433e147de0c5f)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.838673 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.932700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96449207f1a34ef9b5c433e147de0c5f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:59.942303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7d3a4-8a0a-47b3-b26c-126f4cdc76de admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:23:59.947584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96449207f1a34ef9b5c433e147de0c5f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:59.948338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:23:59.948978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:23:59.949131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96449207f1a34ef9b5c433e147de0c5f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:59.949303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:23:59.949558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:23:59.950380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:23:59.952666 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:23:59.978075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96449207f1a34ef9b5c433e147de0c5f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:59.979091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7a3b0308-b1fd-4a26-be4b-f249e79837fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96449207f1a34ef9b5c433e147de0c5f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:23:59.981982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1084/2169] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:23:59 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:23:59.987809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:23:59.989800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:00.000118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1086/2170] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:23:59 2022] DELETE /identity/v3/users/96449207f1a34ef9b5c433e147de0c5f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:00.005472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f28cb1-8e61-4a34-831a-de59cdec84c0 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.005862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f28cb1-8e61-4a34-831a-de59cdec84c0 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.006255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f28cb1-8e61-4a34-831a-de59cdec84c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.006880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b210-2041-44a9-b24a-e549863367be admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:00.034500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-57f28cb1-8e61-4a34-831a-de59cdec84c0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:00.036119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-57f28cb1-8e61-4a34-831a-de59cdec84c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:00.078995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6eed9a-7bc2-45f9-bc6c-9bab6d75316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:00.080459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eed9a-7bc2-45f9-bc6c-9bab6d75316e admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.081217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eed9a-7bc2-45f9-bc6c-9bab6d75316e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.081606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eed9a-7bc2-45f9-bc6c-9bab6d75316e admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.082397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eed9a-7bc2-45f9-bc6c-9bab6d75316e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:00.084299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eed9a-7bc2-45f9-bc6c-9bab6d75316e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:00.096978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1085/2171] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:23:59 2022] GET /identity/v3/roles => generated 1427 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:24:00.105228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6eed9a-7bc2-45f9-bc6c-9bab6d75316e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:00.121035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6eed9a-7bc2-45f9-bc6c-9bab6d75316e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:00.128593 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392799c-835f-4f58-9f4f-ff6c4368244f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:00.129368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392799c-835f-4f58-9f4f-ff6c4368244f admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.129738 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392799c-835f-4f58-9f4f-ff6c4368244f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.129955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392799c-835f-4f58-9f4f-ff6c4368244f admin admin] PATH_INFO: `/v3/projects/edec0a94a057474e9e6e4d865a225922/users/2636f29887d341fda26431d5bbb2cb9e/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.142548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:00.153461 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1087/2172] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:00.160255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392799c-835f-4f58-9f4f-ff6c4368244f admin admin] RBAC: Authorizing `identity:create_grant(project_id=edec0a94a057474e9e6e4d865a225922, user_id=2636f29887d341fda26431d5bbb2cb9e, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:00.163084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392799c-835f-4f58-9f4f-ff6c4368244f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:00.165020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f28cb1-8e61-4a34-831a-de59cdec84c0 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:00.166688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-57f28cb1-8e61-4a34-831a-de59cdec84c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:00.188897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1086/2173] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:00 2022] PUT /identity/v3/projects/edec0a94a057474e9e6e4d865a225922/users/2636f29887d341fda26431d5bbb2cb9e/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:00.189997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392799c-835f-4f58-9f4f-ff6c4368244f admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:00.257511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:00.265456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f28cb1-8e61-4a34-831a-de59cdec84c0 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:00.268946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392799c-835f-4f58-9f4f-ff6c4368244f admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:00.274528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c29403-20c2-472c-ab14-5b953a95f91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:00.275742 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c29403-20c2-472c-ab14-5b953a95f91e admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.276069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c29403-20c2-472c-ab14-5b953a95f91e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.276295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c29403-20c2-472c-ab14-5b953a95f91e admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.277001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c29403-20c2-472c-ab14-5b953a95f91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:00.278687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c29403-20c2-472c-ab14-5b953a95f91e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:00.295720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1087/2174] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:00 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:00.301444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c29403-20c2-472c-ab14-5b953a95f91e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:00.311805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c29403-20c2-472c-ab14-5b953a95f91e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:00.318512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff01b5c-3cf7-46da-8762-e390cc4857fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:00.318888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff01b5c-3cf7-46da-8762-e390cc4857fa admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.319169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff01b5c-3cf7-46da-8762-e390cc4857fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.319439 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff01b5c-3cf7-46da-8762-e390cc4857fa admin admin] PATH_INFO: `/v3/projects/edec0a94a057474e9e6e4d865a225922/users/2636f29887d341fda26431d5bbb2cb9e/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.321516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff01b5c-3cf7-46da-8762-e390cc4857fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=edec0a94a057474e9e6e4d865a225922, user_id=2636f29887d341fda26431d5bbb2cb9e, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:00.324516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff01b5c-3cf7-46da-8762-e390cc4857fa admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:00.348193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1088/2175] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:00 2022] PUT /identity/v3/projects/edec0a94a057474e9e6e4d865a225922/users/2636f29887d341fda26431d5bbb2cb9e/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:00.351451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4521b2-a377-40a8-acb4-18de2696e809 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.351451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4521b2-a377-40a8-acb4-18de2696e809 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.351451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4521b2-a377-40a8-acb4-18de2696e809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.361304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:00.363000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.363000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.363000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.379904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:00.381375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:00.387368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-6f4521b2-a377-40a8-acb4-18de2696e809 None None] MFA Rules not processed for user `2636f29887d341fda26431d5bbb2cb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:00.388324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6f4521b2-a377-40a8-acb4-18de2696e809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:00.400495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1088/2176] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:00.489510 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1089/2177] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:00.498617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:00.501204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:00.618264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ef872-4ceb-40c4-b9e1-af274cdd3288 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:00.699557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:00.700247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.700529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.700847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.738475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:00.740685 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:00.765640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4521b2-a377-40a8-acb4-18de2696e809 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:00.767500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6f4521b2-a377-40a8-acb4-18de2696e809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:00.771965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1089/2178] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:00.844810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:00.847088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:00.858747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5a492-8167-45a9-b359-fc482b3b87f2 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:00.862794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:00.863600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef732e83-f859-4206-a458-38b5c07dd582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:00.864229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732e83-f859-4206-a458-38b5c07dd582 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.864505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732e83-f859-4206-a458-38b5c07dd582 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.864775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732e83-f859-4206-a458-38b5c07dd582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.867462 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ef732e83-f859-4206-a458-38b5c07dd582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:00.871499 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4521b2-a377-40a8-acb4-18de2696e809 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:00.890201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:00.891473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:00.891930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:00.892210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] PATH_INFO: `/v3/projects/c5a29b3102eb43fd94bdd568781904a9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:00.893594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a29b3102eb43fd94bdd568781904a9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:00.895841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:00.955686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef732e83-f859-4206-a458-38b5c07dd582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:00.957251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef732e83-f859-4206-a458-38b5c07dd582 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:00.969052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1090/2179] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:24:00.988241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a29b3102eb43fd94bdd568781904a9', 'request_id': 'req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:00.996316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a29b3102eb43fd94bdd568781904a9', 'request_id': 'req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:01.007753 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a29b3102eb43fd94bdd568781904a9', 'request_id': 'req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:01.023268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] The token cache is being invalidate because project c5a29b3102eb43fd94bdd568781904a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:01.023675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:01.024443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-df7e7a4a-325e-419c-8b1d-7ea7e8d5b902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:01.026901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1090/2180] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:00 2022] DELETE /identity/v3/projects/c5a29b3102eb43fd94bdd568781904a9 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:01.032009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13879b1-9c8c-4036-9d00-7744fca426b5 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.032297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13879b1-9c8c-4036-9d00-7744fca426b5 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.032617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13879b1-9c8c-4036-9d00-7744fca426b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.042448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:01.059518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f13879b1-9c8c-4036-9d00-7744fca426b5 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:01.060736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f13879b1-9c8c-4036-9d00-7744fca426b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:01.155572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1091/2181] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:01.163216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ceea-4265-4fb4-8d17-6a2bc247d7a1 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.163504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ceea-4265-4fb4-8d17-6a2bc247d7a1 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.163777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ceea-4265-4fb4-8d17-6a2bc247d7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.186987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7611ceea-4265-4fb4-8d17-6a2bc247d7a1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:01.197918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7611ceea-4265-4fb4-8d17-6a2bc247d7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:01.203909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1091/2182] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:01.204415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13879b1-9c8c-4036-9d00-7744fca426b5 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:01.216749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13879b1-9c8c-4036-9d00-7744fca426b5 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:01.227421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:01.228412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.228912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.229376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.230956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:24:01.231479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:01.233840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:01.258588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1092/2183] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:24:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:24:01.263485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611ceea-4265-4fb4-8d17-6a2bc247d7a1 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:01.275253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611ceea-4265-4fb4-8d17-6a2bc247d7a1 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:01.285440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abf931-01a8-40c9-b27f-2bd278d4074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:01.285440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf931-01a8-40c9-b27f-2bd278d4074a admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.285440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf931-01a8-40c9-b27f-2bd278d4074a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.285922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf931-01a8-40c9-b27f-2bd278d4074a admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.286721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abf931-01a8-40c9-b27f-2bd278d4074a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:01.289082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abf931-01a8-40c9-b27f-2bd278d4074a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:01.328769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1092/2184] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:01 2022] POST /identity/v3/projects => generated 376 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:01.333345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:01.347422 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d25b-06cd-49ea-8ce4-644c42ed9359 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:01.488932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ceed8-2e50-4ef8-be92-493ad92b55da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:01.490205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ceed8-2e50-4ef8-be92-493ad92b55da admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.490532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ceed8-2e50-4ef8-be92-493ad92b55da admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.490934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ceed8-2e50-4ef8-be92-493ad92b55da admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.492049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ceed8-2e50-4ef8-be92-493ad92b55da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:01.494243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ceed8-2e50-4ef8-be92-493ad92b55da admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:01.520937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1093/2185] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:01 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:24:01.522339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abf931-01a8-40c9-b27f-2bd278d4074a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:01.534144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abf931-01a8-40c9-b27f-2bd278d4074a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:01.540276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec182ae-9880-43c5-82fd-44174740602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:01.541046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec182ae-9880-43c5-82fd-44174740602e admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.541397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec182ae-9880-43c5-82fd-44174740602e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.541756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec182ae-9880-43c5-82fd-44174740602e admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.542541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec182ae-9880-43c5-82fd-44174740602e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:01.544260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec182ae-9880-43c5-82fd-44174740602e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:01.556679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1093/2186] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:01 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:01.560657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ceed8-2e50-4ef8-be92-493ad92b55da admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:01.574709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ceed8-2e50-4ef8-be92-493ad92b55da admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:01.581074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d790739-3a9b-41ce-8eed-8bf92f5b85b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:01.581755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d790739-3a9b-41ce-8eed-8bf92f5b85b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.582096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d790739-3a9b-41ce-8eed-8bf92f5b85b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.582436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d790739-3a9b-41ce-8eed-8bf92f5b85b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.583201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d790739-3a9b-41ce-8eed-8bf92f5b85b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:01.584952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d790739-3a9b-41ce-8eed-8bf92f5b85b0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:01.600649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1094/2187] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:01 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:24:01.602136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec182ae-9880-43c5-82fd-44174740602e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:01.614294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec182ae-9880-43c5-82fd-44174740602e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:01.619746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:01.620421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.620740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.620972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] PATH_INFO: `/v3/projects/8cfbac5e5eab4e719af887179d6bcea8/users/35858dc56fdf4661a8aaffad520fc96f/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.640114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cfbac5e5eab4e719af887179d6bcea8, user_id=35858dc56fdf4661a8aaffad520fc96f, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:01.644749 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:01.660827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1094/2188] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:01 2022] PUT /identity/v3/projects/8cfbac5e5eab4e719af887179d6bcea8/users/35858dc56fdf4661a8aaffad520fc96f/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:01.673470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d07de-4027-4d3c-b44c-accb38190541 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.673806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d07de-4027-4d3c-b44c-accb38190541 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.674029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d07de-4027-4d3c-b44c-accb38190541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.707369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a08d07de-4027-4d3c-b44c-accb38190541 None None] MFA Rules not processed for user `35858dc56fdf4661a8aaffad520fc96f`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:01.708676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a08d07de-4027-4d3c-b44c-accb38190541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:01.793074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:01.799637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1095/2189] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:01.815008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:01.817332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:01.883183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cb2c4-5dbd-4cf3-9392-85babea5a9ac admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:01.887389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769be62c-4e19-41a8-ab14-e3c3bb358310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:01.888222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be62c-4e19-41a8-ab14-e3c3bb358310 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:01.888547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be62c-4e19-41a8-ab14-e3c3bb358310 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:01.888911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be62c-4e19-41a8-ab14-e3c3bb358310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:01.909900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769be62c-4e19-41a8-ab14-e3c3bb358310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:01.911535 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769be62c-4e19-41a8-ab14-e3c3bb358310 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:01.929704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1095/2190] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:24:04.559859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d07de-4027-4d3c-b44c-accb38190541 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:04.562041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a08d07de-4027-4d3c-b44c-accb38190541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:04.692729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d07de-4027-4d3c-b44c-accb38190541 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:04.699375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:04.700337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:04.700702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:04.701088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] PATH_INFO: `/v3/users/209f907510a04bb4950a5a47559de27a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:04.702728 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=209f907510a04bb4950a5a47559de27a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:04.706151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:04.782152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209f907510a04bb4950a5a47559de27a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:04.791973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209f907510a04bb4950a5a47559de27a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:04.792598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209f907510a04bb4950a5a47559de27a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:04.807337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209f907510a04bb4950a5a47559de27a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:04.809821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209f907510a04bb4950a5a47559de27a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:04.824260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1096/2191] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:04 2022] DELETE /identity/v3/users/209f907510a04bb4950a5a47559de27a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:04.829422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0d672-4f05-4d38-ad9e-bad30e96aee1 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:04.829961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0d672-4f05-4d38-ad9e-bad30e96aee1 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:04.829961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0d672-4f05-4d38-ad9e-bad30e96aee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:04.851140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-19f0d672-4f05-4d38-ad9e-bad30e96aee1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:04.852725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-19f0d672-4f05-4d38-ad9e-bad30e96aee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:04.875032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:04.929490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1096/2192] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:04.939691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:04.941863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:05.009915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc54ea-4807-41fd-b4d1-cbdecef4e068 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:05.015046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa752dd-4fdf-4536-a7c4-b172fe3a4590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:05.015791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa752dd-4fdf-4536-a7c4-b172fe3a4590 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:05.016068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa752dd-4fdf-4536-a7c4-b172fe3a4590 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:05.016546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa752dd-4fdf-4536-a7c4-b172fe3a4590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:05.029247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa752dd-4fdf-4536-a7c4-b172fe3a4590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:05.031613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa752dd-4fdf-4536-a7c4-b172fe3a4590 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:05.047299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1097/2193] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:05.371147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0d672-4f05-4d38-ad9e-bad30e96aee1 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:05.372734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-19f0d672-4f05-4d38-ad9e-bad30e96aee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:05.387067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0d672-4f05-4d38-ad9e-bad30e96aee1 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:05.395618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:05.396369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:05.396655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:05.396960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] PATH_INFO: `/v3/projects/77f7467a1e4e44eb8996282966c8d5cd` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:05.399431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f7467a1e4e44eb8996282966c8d5cd)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:05.402414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:05.477594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f7467a1e4e44eb8996282966c8d5cd', 'request_id': 'req-56cab267-6ebc-4e6b-98d1-bffd162be099'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.484452 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f7467a1e4e44eb8996282966c8d5cd', 'request_id': 'req-56cab267-6ebc-4e6b-98d1-bffd162be099'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.494143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f7467a1e4e44eb8996282966c8d5cd', 'request_id': 'req-56cab267-6ebc-4e6b-98d1-bffd162be099'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.507978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] The token cache is being invalidate because project 77f7467a1e4e44eb8996282966c8d5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:05.508477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.509700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.514508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1097/2194] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:05 2022] DELETE /identity/v3/projects/77f7467a1e4e44eb8996282966c8d5cd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:05.515912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa752dd-4fdf-4536-a7c4-b172fe3a4590 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:05.527301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa752dd-4fdf-4536-a7c4-b172fe3a4590 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:05.547338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:05.597274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:05.598131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:05.598469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:05.598955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] PATH_INFO: `/v3/users/2636f29887d341fda26431d5bbb2cb9e` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:05.600195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2636f29887d341fda26431d5bbb2cb9e)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:05.602246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:05.665687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2636f29887d341fda26431d5bbb2cb9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.675143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2636f29887d341fda26431d5bbb2cb9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.676138 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2636f29887d341fda26431d5bbb2cb9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.717481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2636f29887d341fda26431d5bbb2cb9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.718296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0650b92c-9a1d-4909-8d94-049db24f97b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2636f29887d341fda26431d5bbb2cb9e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:05.734017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1098/2195] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:05 2022] DELETE /identity/v3/users/2636f29887d341fda26431d5bbb2cb9e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:06.138253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:06.139791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:06.154727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab267-6ebc-4e6b-98d1-bffd162be099 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:06.216873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:06.217580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.217859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.218133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] PATH_INFO: `/v3/projects/edec0a94a057474e9e6e4d865a225922` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.219330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edec0a94a057474e9e6e4d865a225922)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:06.221509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:06.270787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edec0a94a057474e9e6e4d865a225922', 'request_id': 'req-75747295-4a35-4666-9466-3c2a000151a5'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.271881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:06.277615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edec0a94a057474e9e6e4d865a225922', 'request_id': 'req-75747295-4a35-4666-9466-3c2a000151a5'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.287541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edec0a94a057474e9e6e4d865a225922', 'request_id': 'req-75747295-4a35-4666-9466-3c2a000151a5'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.302273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] The token cache is being invalidate because project edec0a94a057474e9e6e4d865a225922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:06.302566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.303250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.305004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1098/2196] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:06 2022] DELETE /identity/v3/projects/edec0a94a057474e9e6e4d865a225922 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:06.310332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9075c0-d3f7-46a8-b07e-55103221f516 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.310646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9075c0-d3f7-46a8-b07e-55103221f516 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.310927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9075c0-d3f7-46a8-b07e-55103221f516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.321302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:06.323423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:06.334172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-1d9075c0-d3f7-46a8-b07e-55103221f516 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:06.335703 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1d9075c0-d3f7-46a8-b07e-55103221f516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:06.409486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1099/2197] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:06.415108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb833a-3969-4e37-bf0d-8f514431b243 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.415421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb833a-3969-4e37-bf0d-8f514431b243 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.415699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb833a-3969-4e37-bf0d-8f514431b243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.436891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75747295-4a35-4666-9466-3c2a000151a5 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:06.440450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-76eb833a-3969-4e37-bf0d-8f514431b243 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:06.442509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-76eb833a-3969-4e37-bf0d-8f514431b243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:06.443474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:06.444294 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.444564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.444835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] PATH_INFO: `/v3/users/15f1d1302fe34c15bb962e39ecf13469` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.447154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15f1d1302fe34c15bb962e39ecf13469)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:06.449595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:06.457184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1100/2198] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:06.458297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb833a-3969-4e37-bf0d-8f514431b243 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:06.467907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb833a-3969-4e37-bf0d-8f514431b243 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:06.473993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:06.474682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.474942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.475247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.476011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:24:06.477155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:06.478589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:06.496570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1101/2199] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:24:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:24:06.503281 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:06.514548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f11628-09e6-4fec-b13e-e9cb5221e0ca admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:06.521646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bc385-17f1-47d7-bd09-9c5f9cb70e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:06.522465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc385-17f1-47d7-bd09-9c5f9cb70e48 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.522761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc385-17f1-47d7-bd09-9c5f9cb70e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.523042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc385-17f1-47d7-bd09-9c5f9cb70e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.523716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9bc385-17f1-47d7-bd09-9c5f9cb70e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:06.525678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9bc385-17f1-47d7-bd09-9c5f9cb70e48 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:06.529266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15f1d1302fe34c15bb962e39ecf13469'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.540710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15f1d1302fe34c15bb962e39ecf13469'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.551571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:06.553729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15f1d1302fe34c15bb962e39ecf13469'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.556076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15f1d1302fe34c15bb962e39ecf13469'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.566701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1102/2200] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:06 2022] POST /identity/v3/projects => generated 380 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:06.570759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bc385-17f1-47d7-bd09-9c5f9cb70e48 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:06.581017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-aab0aca5-15eb-47e1-9442-8056e252c995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15f1d1302fe34c15bb962e39ecf13469'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:06.588272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1099/2201] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:06 2022] DELETE /identity/v3/users/15f1d1302fe34c15bb962e39ecf13469 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:06.594279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203024c2-f4fc-4e84-9ff8-15d54a09258b None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.594629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203024c2-f4fc-4e84-9ff8-15d54a09258b None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.595282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203024c2-f4fc-4e84-9ff8-15d54a09258b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.630605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-203024c2-f4fc-4e84-9ff8-15d54a09258b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:06.632250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-203024c2-f4fc-4e84-9ff8-15d54a09258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:06.676757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bc385-17f1-47d7-bd09-9c5f9cb70e48 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:06.712947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:06.729865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1100/2202] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:06.742886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203024c2-f4fc-4e84-9ff8-15d54a09258b None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:06.744872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-203024c2-f4fc-4e84-9ff8-15d54a09258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:06.804524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fea8e-45ed-438e-bcfd-4388d1a1cad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:06.804524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fea8e-45ed-438e-bcfd-4388d1a1cad6 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.804524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fea8e-45ed-438e-bcfd-4388d1a1cad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.804524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fea8e-45ed-438e-bcfd-4388d1a1cad6 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.805011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fea8e-45ed-438e-bcfd-4388d1a1cad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:06.805011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fea8e-45ed-438e-bcfd-4388d1a1cad6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:06.829679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1103/2203] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:06 2022] POST /identity/v3/users => generated 291 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:24:06.831421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6beeb-eeb2-475a-9c38-5dbfb630c1f6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.831877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6beeb-eeb2-475a-9c38-5dbfb630c1f6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.832430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6beeb-eeb2-475a-9c38-5dbfb630c1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.833278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203024c2-f4fc-4e84-9ff8-15d54a09258b None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:06.837472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:06.838264 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.838671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.839075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.855012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:06.856749 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:06.861058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-48a6beeb-eeb2-475a-9c38-5dbfb630c1f6 None None] MFA Rules not processed for user `35858dc56fdf4661a8aaffad520fc96f`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:06.862537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-48a6beeb-eeb2-475a-9c38-5dbfb630c1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:06.869612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1101/2204] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:06.870650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:06.872158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:06.888430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4a98b-dedc-4b56-8393-0fad1d0f3f2a service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:06.894722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88e46f-cad3-4986-9013-75893e8e39e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:06.895524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e46f-cad3-4986-9013-75893e8e39e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:06.895909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e46f-cad3-4986-9013-75893e8e39e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:06.896312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e46f-cad3-4986-9013-75893e8e39e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:06.897236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88e46f-cad3-4986-9013-75893e8e39e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:06.899409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88e46f-cad3-4986-9013-75893e8e39e9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:06.939020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1102/2205] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:06 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:06.957727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88e46f-cad3-4986-9013-75893e8e39e9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:07.001506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88e46f-cad3-4986-9013-75893e8e39e9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:07.026249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647d4c1-44c1-47ed-a773-79ece295ac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:07.027366 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d4c1-44c1-47ed-a773-79ece295ac87 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:07.027654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d4c1-44c1-47ed-a773-79ece295ac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:07.027933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d4c1-44c1-47ed-a773-79ece295ac87 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:07.028680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1104/2206] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:07.028965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647d4c1-44c1-47ed-a773-79ece295ac87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:07.034777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647d4c1-44c1-47ed-a773-79ece295ac87 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:07.082467 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1103/2207] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:06 2022] GET /identity/v3/roles => generated 1427 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:07.083791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6beeb-eeb2-475a-9c38-5dbfb630c1f6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:07.128028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6beeb-eeb2-475a-9c38-5dbfb630c1f6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:07.133284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa59b0-3c9b-4605-b5e9-64fa414f6f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:07.134237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa59b0-3c9b-4605-b5e9-64fa414f6f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:07.134630 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa59b0-3c9b-4605-b5e9-64fa414f6f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:07.135021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa59b0-3c9b-4605-b5e9-64fa414f6f7d admin admin] PATH_INFO: `/v3/projects/6af4b9720796434ab45b341a53ad386a/users/b187a1d30fb24267b3e39a1d24960847/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:07.165879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa59b0-3c9b-4605-b5e9-64fa414f6f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af4b9720796434ab45b341a53ad386a, user_id=b187a1d30fb24267b3e39a1d24960847, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:07.168565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa59b0-3c9b-4605-b5e9-64fa414f6f7d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:07.215450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1105/2208] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:07 2022] PUT /identity/v3/projects/6af4b9720796434ab45b341a53ad386a/users/b187a1d30fb24267b3e39a1d24960847/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:07.227413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fcfd0-6366-4401-a27d-c8a171e55a7c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:07.227872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fcfd0-6366-4401-a27d-c8a171e55a7c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:07.228085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fcfd0-6366-4401-a27d-c8a171e55a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:07.281709 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-880fcfd0-6366-4401-a27d-c8a171e55a7c None None] MFA Rules not processed for user `b187a1d30fb24267b3e39a1d24960847`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:07.281709 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-880fcfd0-6366-4401-a27d-c8a171e55a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:07.469451 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1104/2209] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:07.474789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa59b0-3c9b-4605-b5e9-64fa414f6f7d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:07.509318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:07.615651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa59b0-3c9b-4605-b5e9-64fa414f6f7d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:07.626963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fcfd0-6366-4401-a27d-c8a171e55a7c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:07.628664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-880fcfd0-6366-4401-a27d-c8a171e55a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:07.629851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d52936-bf05-4fc1-ac2a-ec678fb8a050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:07.630941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52936-bf05-4fc1-ac2a-ec678fb8a050 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:07.631448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52936-bf05-4fc1-ac2a-ec678fb8a050 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:07.631989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52936-bf05-4fc1-ac2a-ec678fb8a050 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:07.632965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d52936-bf05-4fc1-ac2a-ec678fb8a050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:07.635513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d52936-bf05-4fc1-ac2a-ec678fb8a050 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:07.664135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fcfd0-6366-4401-a27d-c8a171e55a7c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:07.679358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:07.680700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:07.682463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:07.683212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] PATH_INFO: `/v3/projects/077d1444b45e46d8943b4ecaa2523806` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:07.686908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=077d1444b45e46d8943b4ecaa2523806)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:07.691530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:07.711136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1106/2210] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:07 2022] POST /identity/v3/projects => generated 380 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:07.715853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d52936-bf05-4fc1-ac2a-ec678fb8a050 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:07.811501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '077d1444b45e46d8943b4ecaa2523806', 'request_id': 'req-b35d8bec-ba93-4f11-b676-65885ff8fa40'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:07.826101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '077d1444b45e46d8943b4ecaa2523806', 'request_id': 'req-b35d8bec-ba93-4f11-b676-65885ff8fa40'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:07.834515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:07.843208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '077d1444b45e46d8943b4ecaa2523806', 'request_id': 'req-b35d8bec-ba93-4f11-b676-65885ff8fa40'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:07.872385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] The token cache is being invalidate because project 077d1444b45e46d8943b4ecaa2523806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:07.872830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:07.873641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b35d8bec-ba93-4f11-b676-65885ff8fa40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:07.876188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1105/2211] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:07 2022] DELETE /identity/v3/projects/077d1444b45e46d8943b4ecaa2523806 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:07.881204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e18a0c-02e3-4802-8083-14c3e9c0e4e7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:07.881693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e18a0c-02e3-4802-8083-14c3e9c0e4e7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:07.882076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e18a0c-02e3-4802-8083-14c3e9c0e4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:07.911156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-33e18a0c-02e3-4802-8083-14c3e9c0e4e7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:07.912502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-33e18a0c-02e3-4802-8083-14c3e9c0e4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:07.975724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d52936-bf05-4fc1-ac2a-ec678fb8a050 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:07.993462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1106/2212] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:08.003119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b610285-cfd3-4105-b4ad-43266016bfdd None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.004018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b610285-cfd3-4105-b4ad-43266016bfdd None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.004018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b610285-cfd3-4105-b4ad-43266016bfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.028270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2b610285-cfd3-4105-b4ad-43266016bfdd None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:08.030606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2b610285-cfd3-4105-b4ad-43266016bfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:08.052470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1107/2213] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:08.054070 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b610285-cfd3-4105-b4ad-43266016bfdd None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.065394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b610285-cfd3-4105-b4ad-43266016bfdd None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.073101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.074397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.074749 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.075095 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.076374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:24:08.077340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.079740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.100537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1108/2214] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:24:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:24:08.105876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.108155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.109479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.110517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.110885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.111638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.113785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.141400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1107/2215] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:07 2022] POST /identity/v3/users => generated 291 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:08.143250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578a64-e3ca-4435-821d-0e0ff8f7cf85 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.149152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.152684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f343bf-a3fd-4a04-b6ce-48871767e43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.153390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f343bf-a3fd-4a04-b6ce-48871767e43c admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.153966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f343bf-a3fd-4a04-b6ce-48871767e43c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.154095 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f343bf-a3fd-4a04-b6ce-48871767e43c admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.154994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f343bf-a3fd-4a04-b6ce-48871767e43c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.155756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:08.161651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f343bf-a3fd-4a04-b6ce-48871767e43c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.181928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b8e2a-0047-4546-b8af-17958d2bf6ac admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.195386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceef16-9959-4092-88c1-b7af2af52d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.196309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceef16-9959-4092-88c1-b7af2af52d33 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.196740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceef16-9959-4092-88c1-b7af2af52d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.197209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceef16-9959-4092-88c1-b7af2af52d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.198202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ceef16-9959-4092-88c1-b7af2af52d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.200495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ceef16-9959-4092-88c1-b7af2af52d33 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.222157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1108/2216] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:08 2022] GET /identity/v3/roles => generated 1427 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:08.225871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceef16-9959-4092-88c1-b7af2af52d33 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.227622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1109/2217] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:08 2022] POST /identity/v3/projects => generated 384 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:08.232443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f343bf-a3fd-4a04-b6ce-48871767e43c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.242184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceef16-9959-4092-88c1-b7af2af52d33 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.296133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:08.356436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f343bf-a3fd-4a04-b6ce-48871767e43c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.368850 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615025-d6e1-4f4c-9bd1-061f06589472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.369131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615025-d6e1-4f4c-9bd1-061f06589472 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.369300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615025-d6e1-4f4c-9bd1-061f06589472 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.369594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615025-d6e1-4f4c-9bd1-061f06589472 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.370266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615025-d6e1-4f4c-9bd1-061f06589472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.372193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615025-d6e1-4f4c-9bd1-061f06589472 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.379115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1f7e5-08e1-41e1-80dc-6c771a834788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.379747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1f7e5-08e1-41e1-80dc-6c771a834788 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.380022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1f7e5-08e1-41e1-80dc-6c771a834788 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.380295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1f7e5-08e1-41e1-80dc-6c771a834788 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.380994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1f7e5-08e1-41e1-80dc-6c771a834788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.382636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1f7e5-08e1-41e1-80dc-6c771a834788 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.400618 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1109/2218] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:08 2022] GET /identity/v3/roles => generated 1427 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:08.405694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1f7e5-08e1-41e1-80dc-6c771a834788 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.428669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1110/2219] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:08 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:08.430678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615025-d6e1-4f4c-9bd1-061f06589472 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.431244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1f7e5-08e1-41e1-80dc-6c771a834788 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.450343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615025-d6e1-4f4c-9bd1-061f06589472 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.452180 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ef610-fbcc-4d68-a0d3-a96802168896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.454662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ef610-fbcc-4d68-a0d3-a96802168896 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.456104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ef610-fbcc-4d68-a0d3-a96802168896 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.456321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ef610-fbcc-4d68-a0d3-a96802168896 admin admin] PATH_INFO: `/v3/projects/71babb34e0b6434ea4e52030045132cf/users/d72759c486e7416ba4ae877fa6d341e0/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.461815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99395b16-fb2a-498b-bc2a-bf2cf6c0a62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.462586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99395b16-fb2a-498b-bc2a-bf2cf6c0a62c admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.463067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99395b16-fb2a-498b-bc2a-bf2cf6c0a62c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.464424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99395b16-fb2a-498b-bc2a-bf2cf6c0a62c admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.465283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99395b16-fb2a-498b-bc2a-bf2cf6c0a62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.467406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99395b16-fb2a-498b-bc2a-bf2cf6c0a62c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.489764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1111/2220] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:08 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:08.493246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99395b16-fb2a-498b-bc2a-bf2cf6c0a62c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.517378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99395b16-fb2a-498b-bc2a-bf2cf6c0a62c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.522371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ef610-fbcc-4d68-a0d3-a96802168896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71babb34e0b6434ea4e52030045132cf, user_id=d72759c486e7416ba4ae877fa6d341e0, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.523282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a992915-5cac-4396-83ea-8091d40caaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.524031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a992915-5cac-4396-83ea-8091d40caaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.524161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ef610-fbcc-4d68-a0d3-a96802168896 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.524484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a992915-5cac-4396-83ea-8091d40caaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.525072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a992915-5cac-4396-83ea-8091d40caaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.525736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a992915-5cac-4396-83ea-8091d40caaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.527892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a992915-5cac-4396-83ea-8091d40caaa6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.547660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1112/2221] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:08 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:08.549923 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a992915-5cac-4396-83ea-8091d40caaa6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.575486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a992915-5cac-4396-83ea-8091d40caaa6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.581318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1110/2222] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:08 2022] PUT /identity/v3/projects/71babb34e0b6434ea4e52030045132cf/users/d72759c486e7416ba4ae877fa6d341e0/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:08.592742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447a78e-8c60-4628-9072-2ccc3cf9adca None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.593034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447a78e-8c60-4628-9072-2ccc3cf9adca None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.594203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447a78e-8c60-4628-9072-2ccc3cf9adca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.618810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:08.658001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2447a78e-8c60-4628-9072-2ccc3cf9adca None None] MFA Rules not processed for user `d72759c486e7416ba4ae877fa6d341e0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:08.659216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2447a78e-8c60-4628-9072-2ccc3cf9adca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:08.739035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466d080-6de4-446e-bf7a-93a594d92c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:08.740144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d080-6de4-446e-bf7a-93a594d92c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.741259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d080-6de4-446e-bf7a-93a594d92c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.742969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d080-6de4-446e-bf7a-93a594d92c21 admin admin] PATH_INFO: `/v3/projects/260c01e60f5e48ac9420c388a958e0ee/users/91dd37f744a5476488afe91d97a406a1/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.788506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466d080-6de4-446e-bf7a-93a594d92c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260c01e60f5e48ac9420c388a958e0ee, user_id=91dd37f744a5476488afe91d97a406a1, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:08.793962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466d080-6de4-446e-bf7a-93a594d92c21 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:08.820644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1113/2223] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:08 2022] PUT /identity/v3/projects/260c01e60f5e48ac9420c388a958e0ee/users/91dd37f744a5476488afe91d97a406a1/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:08.826594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d575fe4-3ff0-4fb2-b864-b791de2e7b07 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:08.827031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d575fe4-3ff0-4fb2-b864-b791de2e7b07 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:08.827305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d575fe4-3ff0-4fb2-b864-b791de2e7b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:08.839950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1111/2224] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:08.840859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447a78e-8c60-4628-9072-2ccc3cf9adca None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:08.864148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447a78e-8c60-4628-9072-2ccc3cf9adca None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:08.871340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-1d575fe4-3ff0-4fb2-b864-b791de2e7b07 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:08.872664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-1d575fe4-3ff0-4fb2-b864-b791de2e7b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:09.027339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333ba9-48d1-4ea7-957d-f3b17136a8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:09.028666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333ba9-48d1-4ea7-957d-f3b17136a8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:09.030438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333ba9-48d1-4ea7-957d-f3b17136a8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:09.031266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333ba9-48d1-4ea7-957d-f3b17136a8f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:09.036405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333ba9-48d1-4ea7-957d-f3b17136a8f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:09.039584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1114/2225] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:09.045823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333ba9-48d1-4ea7-957d-f3b17136a8f3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:09.053469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d575fe4-3ff0-4fb2-b864-b791de2e7b07 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:09.055319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-1d575fe4-3ff0-4fb2-b864-b791de2e7b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:09.127380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1112/2226] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:08 2022] POST /identity/v3/projects => generated 378 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:09.132301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333ba9-48d1-4ea7-957d-f3b17136a8f3 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:09.164870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:09.166652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d575fe4-3ff0-4fb2-b864-b791de2e7b07 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:09.271824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e80ae-b07f-4db7-bcd5-25e5ac692b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:09.273050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e80ae-b07f-4db7-bcd5-25e5ac692b9a service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:09.273534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e80ae-b07f-4db7-bcd5-25e5ac692b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:09.274141 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e80ae-b07f-4db7-bcd5-25e5ac692b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:09.312792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:09.350346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333ba9-48d1-4ea7-957d-f3b17136a8f3 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:09.362637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c380-d84e-4a2e-a356-aa158f141996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:09.363786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c380-d84e-4a2e-a356-aa158f141996 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:09.364294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c380-d84e-4a2e-a356-aa158f141996 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:09.364789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c380-d84e-4a2e-a356-aa158f141996 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:09.365808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c380-d84e-4a2e-a356-aa158f141996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:09.368370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c380-d84e-4a2e-a356-aa158f141996 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:09.384305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e80ae-b07f-4db7-bcd5-25e5ac692b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:09.385955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e80ae-b07f-4db7-bcd5-25e5ac692b9a service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:09.395302 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1113/2227] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:09 2022] POST /identity/v3/users => generated 296 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:09.397331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1115/2228] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:24:09.398823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c380-d84e-4a2e-a356-aa158f141996 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:09.411185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c380-d84e-4a2e-a356-aa158f141996 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:09.422507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a5146-769e-44d1-811f-c1ef7906404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:09.423381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a5146-769e-44d1-811f-c1ef7906404c admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:09.423856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a5146-769e-44d1-811f-c1ef7906404c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:09.424289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a5146-769e-44d1-811f-c1ef7906404c admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:09.425342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a5146-769e-44d1-811f-c1ef7906404c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:09.427199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a5146-769e-44d1-811f-c1ef7906404c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:09.441934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1114/2229] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:09 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:24:09.445808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e80ae-b07f-4db7-bcd5-25e5ac692b9a service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:09.461373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e80ae-b07f-4db7-bcd5-25e5ac692b9a service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:09.467974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acebdc5-44c6-4a89-9304-875f07833d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:09.469321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acebdc5-44c6-4a89-9304-875f07833d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:09.469637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acebdc5-44c6-4a89-9304-875f07833d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:09.469910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acebdc5-44c6-4a89-9304-875f07833d2a admin admin] PATH_INFO: `/v3/projects/8bf0545da60343a7a7632678a85867a1/users/cf2ecf35f0a24d829d2fd420c79671e9/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:09.488429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acebdc5-44c6-4a89-9304-875f07833d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf0545da60343a7a7632678a85867a1, user_id=cf2ecf35f0a24d829d2fd420c79671e9, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:09.491458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acebdc5-44c6-4a89-9304-875f07833d2a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:09.513846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1116/2230] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:09 2022] PUT /identity/v3/projects/8bf0545da60343a7a7632678a85867a1/users/cf2ecf35f0a24d829d2fd420c79671e9/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:09.521732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a5146-769e-44d1-811f-c1ef7906404c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:09.534905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a5146-769e-44d1-811f-c1ef7906404c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:09.672895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05efcb9a-5e9c-403c-930a-deac65b0c38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:09.674765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efcb9a-5e9c-403c-930a-deac65b0c38d admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:09.675356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efcb9a-5e9c-403c-930a-deac65b0c38d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:09.676711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efcb9a-5e9c-403c-930a-deac65b0c38d admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:09.677634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05efcb9a-5e9c-403c-930a-deac65b0c38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:09.679484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05efcb9a-5e9c-403c-930a-deac65b0c38d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:09.693473 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1115/2231] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:09 2022] GET /identity/v3/roles => generated 1427 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:09.697135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acebdc5-44c6-4a89-9304-875f07833d2a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:09.709384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acebdc5-44c6-4a89-9304-875f07833d2a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:09.715621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:09.716450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:09.716833 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:09.717210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] PATH_INFO: `/v3/projects/8bf0545da60343a7a7632678a85867a1/users/cf2ecf35f0a24d829d2fd420c79671e9/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:09.720755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf0545da60343a7a7632678a85867a1, user_id=cf2ecf35f0a24d829d2fd420c79671e9, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:09.723627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:09.740623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1117/2232] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:09 2022] PUT /identity/v3/projects/8bf0545da60343a7a7632678a85867a1/users/cf2ecf35f0a24d829d2fd420c79671e9/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:09.752844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811fd10-1567-40c7-9349-be278d8be993 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:09.753192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811fd10-1567-40c7-9349-be278d8be993 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:09.753590 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811fd10-1567-40c7-9349-be278d8be993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:09.798706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a811fd10-1567-40c7-9349-be278d8be993 None None] MFA Rules not processed for user `cf2ecf35f0a24d829d2fd420c79671e9`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:09.800458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a811fd10-1567-40c7-9349-be278d8be993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:09.933336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1116/2233] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:09.950102 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:09.951825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:10.026548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547bee-5841-4d23-b6b0-d1fe5b89414c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:10.031128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eb5d8-c601-46b7-b9d2-326af4e59755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:10.032113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb5d8-c601-46b7-b9d2-326af4e59755 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:10.032399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb5d8-c601-46b7-b9d2-326af4e59755 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:10.032671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb5d8-c601-46b7-b9d2-326af4e59755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:10.044259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8eb5d8-c601-46b7-b9d2-326af4e59755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:10.045866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8eb5d8-c601-46b7-b9d2-326af4e59755 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:10.060869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1118/2234] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:10.157827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811fd10-1567-40c7-9349-be278d8be993 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:10.159262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a811fd10-1567-40c7-9349-be278d8be993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:10.172592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811fd10-1567-40c7-9349-be278d8be993 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:10.176742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:10.177267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:10.177488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:10.177710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:10.179552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:10.206951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:10.308472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:10.310175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:10.368119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1117/2235] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:10.706780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504696a7-2286-4a8c-b65a-b1973285e099 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:10.707355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504696a7-2286-4a8c-b65a-b1973285e099 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:10.707355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504696a7-2286-4a8c-b65a-b1973285e099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:10.779662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-504696a7-2286-4a8c-b65a-b1973285e099 None None] MFA Rules not processed for user `91dd37f744a5476488afe91d97a406a1`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:10.783226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-504696a7-2286-4a8c-b65a-b1973285e099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:10.923083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1119/2236] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:10.936367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:10.950998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92561ee1-82ad-4519-bbec-3e6cac0deb26 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:10.961992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba6f31-66b5-4a24-8000-0793c9adc57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:10.962639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba6f31-66b5-4a24-8000-0793c9adc57c service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:10.962925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba6f31-66b5-4a24-8000-0793c9adc57c service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:10.963479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba6f31-66b5-4a24-8000-0793c9adc57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:11.003498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba6f31-66b5-4a24-8000-0793c9adc57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:11.007401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba6f31-66b5-4a24-8000-0793c9adc57c service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:11.035236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1118/2237] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:13.610059 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504696a7-2286-4a8c-b65a-b1973285e099 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:13.613321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-504696a7-2286-4a8c-b65a-b1973285e099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:13.619236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:13.706761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504696a7-2286-4a8c-b65a-b1973285e099 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:13.715028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:13.715429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:13.715949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:13.716300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] PATH_INFO: `/v3/users/3c3b49fd331c4e3f90a8080e8e47aefd` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:13.719634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3b49fd331c4e3f90a8080e8e47aefd)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:13.721807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:13.795187 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3b49fd331c4e3f90a8080e8e47aefd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:13.803797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3b49fd331c4e3f90a8080e8e47aefd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:13.815403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3b49fd331c4e3f90a8080e8e47aefd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:13.816584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3b49fd331c4e3f90a8080e8e47aefd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:13.833366 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3b49fd331c4e3f90a8080e8e47aefd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:13.843965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1120/2238] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:13 2022] DELETE /identity/v3/users/3c3b49fd331c4e3f90a8080e8e47aefd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:13.848236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a56f-afa2-43f3-91f3-f6d7002f7309 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:13.848770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a56f-afa2-43f3-91f3-f6d7002f7309 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:13.849076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a56f-afa2-43f3-91f3-f6d7002f7309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:13.872056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0591a56f-afa2-43f3-91f3-f6d7002f7309 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:13.873638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0591a56f-afa2-43f3-91f3-f6d7002f7309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:13.970913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1119/2239] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:13.980482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:13.982247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:14.008928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa4c42-491e-47cf-a6db-81820125799d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:14.009725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa4c42-491e-47cf-a6db-81820125799d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:14.010253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa4c42-491e-47cf-a6db-81820125799d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:14.052419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-42aa4c42-491e-47cf-a6db-81820125799d None None] MFA Rules not processed for user `91dd37f744a5476488afe91d97a406a1`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:14.059048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-42aa4c42-491e-47cf-a6db-81820125799d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:14.126164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6798fc-86c3-46f0-a45d-369031cc702d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:14.131035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:14.131653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:14.131931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:14.132202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:14.146577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:14.148025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:14.164940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1121/2240] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:14.167044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:14.168961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:14.178384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1120/2241] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:14.185848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948890a4-52c0-43c6-b26a-37bd43b87de4 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:14.193162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:14.193983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:14.194356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:14.194730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] PATH_INFO: `/v3/users/b187a1d30fb24267b3e39a1d24960847` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:14.196051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b187a1d30fb24267b3e39a1d24960847)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:14.198184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:14.238384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:14.286038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b187a1d30fb24267b3e39a1d24960847'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.294818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b187a1d30fb24267b3e39a1d24960847'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.306667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b187a1d30fb24267b3e39a1d24960847'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.307636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b187a1d30fb24267b3e39a1d24960847'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.325789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b187a1d30fb24267b3e39a1d24960847'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.339316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1122/2242] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:14 2022] DELETE /identity/v3/users/b187a1d30fb24267b3e39a1d24960847 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:14.345951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af757be-e621-4504-81d8-3e782b284f9c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:14.346902 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af757be-e621-4504-81d8-3e782b284f9c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:14.347188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af757be-e621-4504-81d8-3e782b284f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:14.372759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8af757be-e621-4504-81d8-3e782b284f9c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:14.376507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8af757be-e621-4504-81d8-3e782b284f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:14.448947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:14.519536 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1121/2243] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:14.557925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:14.559655 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:14.632846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4a21f-ceb1-44bb-837a-98e926f7b7d2 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:14.637709 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7808aa-e0fb-4ca6-ab56-7e93827f5bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:14.639006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7808aa-e0fb-4ca6-ab56-7e93827f5bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:14.639332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7808aa-e0fb-4ca6-ab56-7e93827f5bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:14.640323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7808aa-e0fb-4ca6-ab56-7e93827f5bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:14.651602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7808aa-e0fb-4ca6-ab56-7e93827f5bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:14.653176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7808aa-e0fb-4ca6-ab56-7e93827f5bb3 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:14.668534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1123/2244] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:14.737212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af757be-e621-4504-81d8-3e782b284f9c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:14.738693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8af757be-e621-4504-81d8-3e782b284f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:14.788819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af757be-e621-4504-81d8-3e782b284f9c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:14.816231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:14.816971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:14.817734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:14.818023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] PATH_INFO: `/v3/projects/64564daee3084f369f093513059033eb` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:14.820015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] RBAC: Authorizing `identity:delete_project(project_id=64564daee3084f369f093513059033eb)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:14.825098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:14.931188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64564daee3084f369f093513059033eb', 'request_id': 'req-d93ff03e-d59e-4d21-89f8-13a1d4b623be'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.944862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64564daee3084f369f093513059033eb', 'request_id': 'req-d93ff03e-d59e-4d21-89f8-13a1d4b623be'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.952584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64564daee3084f369f093513059033eb', 'request_id': 'req-d93ff03e-d59e-4d21-89f8-13a1d4b623be'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.974956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] The token cache is being invalidate because project 64564daee3084f369f093513059033eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:14.975176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.976017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d93ff03e-d59e-4d21-89f8-13a1d4b623be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:14.978104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1122/2245] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:14 2022] DELETE /identity/v3/projects/64564daee3084f369f093513059033eb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:14.983323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90ebd0-6e2d-45a9-9135-b63ef4dfbae0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:14.984317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90ebd0-6e2d-45a9-9135-b63ef4dfbae0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:14.984730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90ebd0-6e2d-45a9-9135-b63ef4dfbae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.010909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8e90ebd0-6e2d-45a9-9135-b63ef4dfbae0 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:15.012054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8e90ebd0-6e2d-45a9-9135-b63ef4dfbae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:15.110046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1124/2246] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:15.122949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9a8b6-1e15-4065-8f40-75f6568cf99d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:15.123352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9a8b6-1e15-4065-8f40-75f6568cf99d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:15.123632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9a8b6-1e15-4065-8f40-75f6568cf99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.160702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-33c9a8b6-1e15-4065-8f40-75f6568cf99d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:15.160702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-33c9a8b6-1e15-4065-8f40-75f6568cf99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:15.173362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1123/2247] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:15.174776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90ebd0-6e2d-45a9-9135-b63ef4dfbae0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:15.206076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90ebd0-6e2d-45a9-9135-b63ef4dfbae0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:15.216117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:15.217476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:15.218663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:15.219050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.223490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:24:15.223921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:15.226034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:15.247026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1125/2248] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:24:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:24:15.248504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9a8b6-1e15-4065-8f40-75f6568cf99d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:15.267459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9a8b6-1e15-4065-8f40-75f6568cf99d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:15.277232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadce733-a28b-4010-bb50-781c82d384d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:15.278151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadce733-a28b-4010-bb50-781c82d384d3 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:15.278535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadce733-a28b-4010-bb50-781c82d384d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:15.278913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadce733-a28b-4010-bb50-781c82d384d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.279723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadce733-a28b-4010-bb50-781c82d384d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:15.282855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadce733-a28b-4010-bb50-781c82d384d3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:15.339470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1124/2249] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:15 2022] POST /identity/v3/projects => generated 374 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:15.341013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:15.357316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591042e-c22f-47bd-8c69-e1872cfb2465 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:15.393962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:15.478204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f933ab-3cd7-4cf4-af94-26fecc217866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:15.478973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f933ab-3cd7-4cf4-af94-26fecc217866 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:15.479328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f933ab-3cd7-4cf4-af94-26fecc217866 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:15.479674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f933ab-3cd7-4cf4-af94-26fecc217866 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.480427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f933ab-3cd7-4cf4-af94-26fecc217866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:15.483888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f933ab-3cd7-4cf4-af94-26fecc217866 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:15.537538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1126/2250] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:15 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:15.539665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadce733-a28b-4010-bb50-781c82d384d3 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:15.552155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadce733-a28b-4010-bb50-781c82d384d3 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:15.557273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de843a-b0b5-432e-b124-f392e7a6c937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:15.557993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de843a-b0b5-432e-b124-f392e7a6c937 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:15.558406 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de843a-b0b5-432e-b124-f392e7a6c937 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:15.559190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de843a-b0b5-432e-b124-f392e7a6c937 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.560522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de843a-b0b5-432e-b124-f392e7a6c937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:15.566260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de843a-b0b5-432e-b124-f392e7a6c937 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:15.629110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1125/2251] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:15 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:15.629826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f933ab-3cd7-4cf4-af94-26fecc217866 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:15.641631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f933ab-3cd7-4cf4-af94-26fecc217866 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:15.649471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cfd3c-f558-470a-8aa6-852ac81597cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:15.650573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cfd3c-f558-470a-8aa6-852ac81597cf admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:15.650950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cfd3c-f558-470a-8aa6-852ac81597cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:15.651338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cfd3c-f558-470a-8aa6-852ac81597cf admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.653703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cfd3c-f558-470a-8aa6-852ac81597cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:15.664492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cfd3c-f558-470a-8aa6-852ac81597cf admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:15.678866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1127/2252] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:15 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:24:15.683979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de843a-b0b5-432e-b124-f392e7a6c937 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:15.698011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de843a-b0b5-432e-b124-f392e7a6c937 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:15.704065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:15.704979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:15.705923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:15.706659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] PATH_INFO: `/v3/projects/6d5a12bbc14c42bcb3c8e4a4dffcd04a/users/8ac4820f652e4eec8bb7454ccededc12/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.748850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5a12bbc14c42bcb3c8e4a4dffcd04a, user_id=8ac4820f652e4eec8bb7454ccededc12, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:15.755714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:15.777779 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1126/2253] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:15 2022] PUT /identity/v3/projects/6d5a12bbc14c42bcb3c8e4a4dffcd04a/users/8ac4820f652e4eec8bb7454ccededc12/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:15.791206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7399b5-ebc6-4cd6-8055-a07daba1728f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:15.791520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7399b5-ebc6-4cd6-8055-a07daba1728f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:15.791802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7399b5-ebc6-4cd6-8055-a07daba1728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:15.880702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-de7399b5-ebc6-4cd6-8055-a07daba1728f None None] MFA Rules not processed for user `8ac4820f652e4eec8bb7454ccededc12`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:15.882696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-de7399b5-ebc6-4cd6-8055-a07daba1728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:15.985599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1128/2254] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:15.988977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:15.992897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:16.081013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1df3a9-b29d-4442-b295-4576cb4f0e93 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:16.086569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:16.087505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:16.088032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:16.089075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:16.104205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:16.106140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:16.123069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1127/2255] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:16.160244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7399b5-ebc6-4cd6-8055-a07daba1728f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:16.239533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7399b5-ebc6-4cd6-8055-a07daba1728f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:16.246736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:16.247498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:16.247783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:16.248055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] PATH_INFO: `/v3/users/8ac4820f652e4eec8bb7454ccededc12` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:16.249836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac4820f652e4eec8bb7454ccededc12)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:16.251862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:16.263743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:16.266629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:16.290626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f9589-d97d-434e-9ec6-b8e8ce7ecd29 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:16.304548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:16.305858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:16.306736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:16.308745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] PATH_INFO: `/v3/projects/6af4b9720796434ab45b341a53ad386a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:16.310037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af4b9720796434ab45b341a53ad386a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:16.319036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:16.320845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:16.349165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:16.373922 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac4820f652e4eec8bb7454ccededc12'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.390144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac4820f652e4eec8bb7454ccededc12'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.400208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac4820f652e4eec8bb7454ccededc12'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.400931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac4820f652e4eec8bb7454ccededc12'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.420537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b7b4d13b-9f9b-44da-8d68-f39fbb6e49b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac4820f652e4eec8bb7454ccededc12'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.430864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af4b9720796434ab45b341a53ad386a', 'request_id': 'req-10df81e2-d5d6-4238-8fc5-e231f11fdf80'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.432067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1129/2256] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:16 2022] DELETE /identity/v3/users/8ac4820f652e4eec8bb7454ccededc12 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:16.433446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fbd9-40b1-4053-bdb4-b228546678c3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:16.433744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fbd9-40b1-4053-bdb4-b228546678c3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:16.434020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fbd9-40b1-4053-bdb4-b228546678c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:16.445220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af4b9720796434ab45b341a53ad386a', 'request_id': 'req-10df81e2-d5d6-4238-8fc5-e231f11fdf80'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.458851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af4b9720796434ab45b341a53ad386a', 'request_id': 'req-10df81e2-d5d6-4238-8fc5-e231f11fdf80'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.462232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-4e49fbd9-40b1-4053-bdb4-b228546678c3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:16.462891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4e49fbd9-40b1-4053-bdb4-b228546678c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:16.478552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] The token cache is being invalidate because project 6af4b9720796434ab45b341a53ad386a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:16.479105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.480046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.487358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1128/2257] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:16 2022] DELETE /identity/v3/projects/6af4b9720796434ab45b341a53ad386a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:16.492000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:16.497840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:16.590516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1130/2258] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:16.603911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fbd9-40b1-4053-bdb4-b228546678c3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:16.606813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4e49fbd9-40b1-4053-bdb4-b228546678c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:16.617669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df81e2-d5d6-4238-8fc5-e231f11fdf80 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:16.623120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:16.624110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:16.624711 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:16.625148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] PATH_INFO: `/v3/users/d72759c486e7416ba4ae877fa6d341e0` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:16.626578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d72759c486e7416ba4ae877fa6d341e0)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:16.628969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:16.695477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fbd9-40b1-4053-bdb4-b228546678c3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:16.700120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:16.700881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:16.701317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:16.701752 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:16.713134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:16.714716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:16.716212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd72759c486e7416ba4ae877fa6d341e0'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.728420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd72759c486e7416ba4ae877fa6d341e0'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.729185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd72759c486e7416ba4ae877fa6d341e0'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.730369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1131/2259] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:24:16.746064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd72759c486e7416ba4ae877fa6d341e0'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.746989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd72759c486e7416ba4ae877fa6d341e0'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:16.765909 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1129/2260] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:16 2022] DELETE /identity/v3/users/d72759c486e7416ba4ae877fa6d341e0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:17.168941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:17.170411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:17.188870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bc65c-bb4d-49b4-8124-c14a40ec37e5 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:17.291834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:17.302050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:17.302717 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:17.303071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:17.303388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] PATH_INFO: `/v3/projects/6d5a12bbc14c42bcb3c8e4a4dffcd04a` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:17.305739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d5a12bbc14c42bcb3c8e4a4dffcd04a)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:17.308538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:17.390560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:17.391907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:17.419119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5a12bbc14c42bcb3c8e4a4dffcd04a', 'request_id': 'req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.426831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40c4b5-8aa3-42c1-9772-89ff18e5990b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:17.444269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5a12bbc14c42bcb3c8e4a4dffcd04a', 'request_id': 'req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.447384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:17.484379 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5a12bbc14c42bcb3c8e4a4dffcd04a', 'request_id': 'req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.506321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] The token cache is being invalidate because project 6d5a12bbc14c42bcb3c8e4a4dffcd04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:17.507981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.509169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-3dc44b7c-7570-4eb6-90f0-a78480d2a76c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.515211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1132/2261] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:17 2022] DELETE /identity/v3/projects/6d5a12bbc14c42bcb3c8e4a4dffcd04a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:17.520115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63ff9-79ea-4253-8940-60984e50b325 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:17.520667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63ff9-79ea-4253-8940-60984e50b325 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:17.521182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63ff9-79ea-4253-8940-60984e50b325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:17.571468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e5b63ff9-79ea-4253-8940-60984e50b325 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:17.574216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e5b63ff9-79ea-4253-8940-60984e50b325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:17.601727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:17.602816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:17.603867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:17.604363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] PATH_INFO: `/v3/projects/71babb34e0b6434ea4e52030045132cf` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:17.606822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71babb34e0b6434ea4e52030045132cf)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:17.611613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:17.688927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1133/2262] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:17.691978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71babb34e0b6434ea4e52030045132cf', 'request_id': 'req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.696132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17cad6-57de-4703-afe5-f775b6a1bd96 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:17.697813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17cad6-57de-4703-afe5-f775b6a1bd96 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:17.698536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17cad6-57de-4703-afe5-f775b6a1bd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:17.704588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71babb34e0b6434ea4e52030045132cf', 'request_id': 'req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.716069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71babb34e0b6434ea4e52030045132cf', 'request_id': 'req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.739753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] The token cache is being invalidate because project 71babb34e0b6434ea4e52030045132cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:17.740266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.741843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:17.743143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ad17cad6-57de-4703-afe5-f775b6a1bd96 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:17.746757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1130/2263] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:17 2022] DELETE /identity/v3/projects/71babb34e0b6434ea4e52030045132cf => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:17.746914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ad17cad6-57de-4703-afe5-f775b6a1bd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:17.748912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:17.752856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:17.888991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1134/2264] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:17.891611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17cad6-57de-4703-afe5-f775b6a1bd96 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:17.901520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0dd11-0a53-43ae-8f9f-94d5115cb191 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:17.910058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:17.911385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:17.911677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:17.912093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] PATH_INFO: `/v3/users/cf2ecf35f0a24d829d2fd420c79671e9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:17.914445 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf2ecf35f0a24d829d2fd420c79671e9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:17.914899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17cad6-57de-4703-afe5-f775b6a1bd96 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:17.920094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:17.932589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:17.934390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:17.934774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:17.935144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:17.936469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:24:17.936469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:17.938871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:17.956514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1135/2265] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:24:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:24:17.959045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:17.979274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423ee22-6b86-49b4-8a19-a3b7ec5a7d04 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:17.989511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:17.991288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:17.992267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:17.992770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:17.994564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:18.000249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:18.039644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:18.047455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:18.050616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf2ecf35f0a24d829d2fd420c79671e9'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:18.136183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf2ecf35f0a24d829d2fd420c79671e9'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:18.136183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf2ecf35f0a24d829d2fd420c79671e9'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:18.137265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1136/2266] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:17 2022] POST /identity/v3/projects => generated 376 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:18.137368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf2ecf35f0a24d829d2fd420c79671e9'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:18.137997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:18.163130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf2ecf35f0a24d829d2fd420c79671e9'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:18.167546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:18.222275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1131/2267] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:17 2022] DELETE /identity/v3/users/cf2ecf35f0a24d829d2fd420c79671e9 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:18.287326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3adb-0ef7-4113-b37a-9fdf04d19502 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:18.379129 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f73cc-52c7-419e-bb9e-3aee0c65047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:18.379771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f73cc-52c7-419e-bb9e-3aee0c65047e admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:18.380057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f73cc-52c7-419e-bb9e-3aee0c65047e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:18.380331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f73cc-52c7-419e-bb9e-3aee0c65047e admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:18.381053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f73cc-52c7-419e-bb9e-3aee0c65047e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:18.383518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f73cc-52c7-419e-bb9e-3aee0c65047e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:18.407973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1137/2268] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:18 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:24:18.412348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:18.432363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37803539-871b-45b7-80af-b9fa70cf1ba9 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:18.443327 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24941d5e-1c87-4745-9d7c-e7c5dabbe386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:18.445024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24941d5e-1c87-4745-9d7c-e7c5dabbe386 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:18.445849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24941d5e-1c87-4745-9d7c-e7c5dabbe386 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:18.446579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24941d5e-1c87-4745-9d7c-e7c5dabbe386 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:18.450423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24941d5e-1c87-4745-9d7c-e7c5dabbe386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:18.454758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24941d5e-1c87-4745-9d7c-e7c5dabbe386 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:18.477597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1132/2269] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:18 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:18.478663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f73cc-52c7-419e-bb9e-3aee0c65047e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:18.493228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f73cc-52c7-419e-bb9e-3aee0c65047e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:18.498395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc999d-df6f-464d-a7e4-b1c25ecadc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:18.500282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc999d-df6f-464d-a7e4-b1c25ecadc99 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:18.500282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc999d-df6f-464d-a7e4-b1c25ecadc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:18.500557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc999d-df6f-464d-a7e4-b1c25ecadc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:18.501549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc999d-df6f-464d-a7e4-b1c25ecadc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:18.503238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc999d-df6f-464d-a7e4-b1c25ecadc99 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:18.529253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24941d5e-1c87-4745-9d7c-e7c5dabbe386 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:18.539828 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1138/2270] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:18 2022] GET /identity/v3/roles => generated 1427 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:24:18.542007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24941d5e-1c87-4745-9d7c-e7c5dabbe386 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:18.551665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:18.552543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:18.552928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:18.553309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] PATH_INFO: `/v3/projects/467075e54e354e38b220629fbf5df535/users/755df1ae00c6484780dcfba580935d28/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:18.581897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467075e54e354e38b220629fbf5df535, user_id=755df1ae00c6484780dcfba580935d28, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:18.584768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:18.625370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1133/2271] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:18 2022] PUT /identity/v3/projects/467075e54e354e38b220629fbf5df535/users/755df1ae00c6484780dcfba580935d28/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:18.627310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331ade-3f92-468b-a926-837fc129453a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:18.627950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331ade-3f92-468b-a926-837fc129453a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:18.629256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331ade-3f92-468b-a926-837fc129453a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:18.689935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:18.693245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:18.712444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5f331ade-3f92-468b-a926-837fc129453a None None] MFA Rules not processed for user `755df1ae00c6484780dcfba580935d28`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:18.726126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5f331ade-3f92-468b-a926-837fc129453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:18.936174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755be70-01b4-4ff2-8001-347d3bd35c36 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:18.942389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1139/2272] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:18.953739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f331ade-3f92-468b-a926-837fc129453a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:18.959307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:18.959944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5f331ade-3f92-468b-a926-837fc129453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:18.960072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:18.960251 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:18.960855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] PATH_INFO: `/v3/projects/8bf0545da60343a7a7632678a85867a1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:18.962917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf0545da60343a7a7632678a85867a1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:18.964875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:18.995292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:19.142250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f331ade-3f92-468b-a926-837fc129453a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:19.156357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf0545da60343a7a7632678a85867a1', 'request_id': 'req-844a2c7a-6c23-4fdd-9019-a22a583d3308'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:19.200412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf0545da60343a7a7632678a85867a1', 'request_id': 'req-844a2c7a-6c23-4fdd-9019-a22a583d3308'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:19.222160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf0545da60343a7a7632678a85867a1', 'request_id': 'req-844a2c7a-6c23-4fdd-9019-a22a583d3308'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:19.254747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] The token cache is being invalidate because project 8bf0545da60343a7a7632678a85867a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:19.255183 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:19.255728 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-844a2c7a-6c23-4fdd-9019-a22a583d3308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:19.261679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1134/2273] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:18 2022] DELETE /identity/v3/projects/8bf0545da60343a7a7632678a85867a1 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:19.263531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768a42c-2716-424d-85a2-66b3a2fee60f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:19.263531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768a42c-2716-424d-85a2-66b3a2fee60f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:19.263744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768a42c-2716-424d-85a2-66b3a2fee60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:19.287271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:19.287960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:19.288245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:19.290939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:19.297588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:19.322955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-f768a42c-2716-424d-85a2-66b3a2fee60f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:19.324584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f768a42c-2716-424d-85a2-66b3a2fee60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:19.345010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:19.483199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:19.486258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:19.498423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1135/2274] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:19.505716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1140/2275] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:19.508812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8362b0-cc4b-4a5c-aee2-088766c3acba None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:19.509103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8362b0-cc4b-4a5c-aee2-088766c3acba None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:19.509380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8362b0-cc4b-4a5c-aee2-088766c3acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:19.531066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-dc8362b0-cc4b-4a5c-aee2-088766c3acba None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:19.532336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-dc8362b0-cc4b-4a5c-aee2-088766c3acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:19.540893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1136/2276] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:19.545147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:19.561041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdc79-f4fc-4648-a061-d5234cc109cd service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:19.566736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:19.567367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:19.567645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:19.568027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:19.569017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:24:19.569525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:19.571290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:19.589517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1141/2277] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:24:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:24:19.590599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8362b0-cc4b-4a5c-aee2-088766c3acba None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:19.603249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8362b0-cc4b-4a5c-aee2-088766c3acba None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:19.613820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29ffb4-d4c3-4e31-948c-17c1e2ce72f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:19.614650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29ffb4-d4c3-4e31-948c-17c1e2ce72f6 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:19.615039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29ffb4-d4c3-4e31-948c-17c1e2ce72f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:19.616326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29ffb4-d4c3-4e31-948c-17c1e2ce72f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:19.616326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29ffb4-d4c3-4e31-948c-17c1e2ce72f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:19.618735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29ffb4-d4c3-4e31-948c-17c1e2ce72f6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:19.682585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1137/2278] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:19 2022] POST /identity/v3/projects => generated 382 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:19.685018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:19.700609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30100e34-f324-4385-8f1e-230b3a1dc0ca admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:19.802139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e786dd-2d80-4ced-92a8-2808d6c2df22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:19.803564 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e786dd-2d80-4ced-92a8-2808d6c2df22 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:19.804024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e786dd-2d80-4ced-92a8-2808d6c2df22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:19.804484 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e786dd-2d80-4ced-92a8-2808d6c2df22 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:19.805364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e786dd-2d80-4ced-92a8-2808d6c2df22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:19.809180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e786dd-2d80-4ced-92a8-2808d6c2df22 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:19.822437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:19.843979 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1142/2279] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:19 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:19.846290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29ffb4-d4c3-4e31-948c-17c1e2ce72f6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:19.873383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29ffb4-d4c3-4e31-948c-17c1e2ce72f6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:19.889852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eef1e-54cc-4b0b-ac6f-494d40c24e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:19.890557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eef1e-54cc-4b0b-ac6f-494d40c24e72 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:19.890931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eef1e-54cc-4b0b-ac6f-494d40c24e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:19.891310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eef1e-54cc-4b0b-ac6f-494d40c24e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:19.891929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31eef1e-54cc-4b0b-ac6f-494d40c24e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:19.897168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31eef1e-54cc-4b0b-ac6f-494d40c24e72 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:19.915935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1138/2280] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:19 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:19.917310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e786dd-2d80-4ced-92a8-2808d6c2df22 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:19.942491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e786dd-2d80-4ced-92a8-2808d6c2df22 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:19.955543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4da27a-eead-47b3-b2e3-e0f23baa863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:19.956132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4da27a-eead-47b3-b2e3-e0f23baa863e admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:19.956444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4da27a-eead-47b3-b2e3-e0f23baa863e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:19.959252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4da27a-eead-47b3-b2e3-e0f23baa863e admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:19.959974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4da27a-eead-47b3-b2e3-e0f23baa863e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:19.963807 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4da27a-eead-47b3-b2e3-e0f23baa863e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:19.991361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1143/2281] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:19 2022] GET /identity/v3/roles => generated 1427 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:19.992539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eef1e-54cc-4b0b-ac6f-494d40c24e72 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:20.004950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eef1e-54cc-4b0b-ac6f-494d40c24e72 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:20.040530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576448-c5f5-46ab-b8e5-89d286b0d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:20.041237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576448-c5f5-46ab-b8e5-89d286b0d26e admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:20.041555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576448-c5f5-46ab-b8e5-89d286b0d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:20.041861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576448-c5f5-46ab-b8e5-89d286b0d26e admin admin] PATH_INFO: `/v3/projects/9f46074a3077408cba379bc9e790c660/users/af2b5817e77f4ce7ad0791f5a6ea64af/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:20.112439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f576448-c5f5-46ab-b8e5-89d286b0d26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f46074a3077408cba379bc9e790c660, user_id=af2b5817e77f4ce7ad0791f5a6ea64af, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:20.118361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f576448-c5f5-46ab-b8e5-89d286b0d26e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:20.165459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1139/2282] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:19 2022] PUT /identity/v3/projects/9f46074a3077408cba379bc9e790c660/users/af2b5817e77f4ce7ad0791f5a6ea64af/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:20.172323 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719d762-9bcb-4a70-9610-bc075430bbaf None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:20.172730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719d762-9bcb-4a70-9610-bc075430bbaf None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:20.172876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719d762-9bcb-4a70-9610-bc075430bbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:20.224447 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2719d762-9bcb-4a70-9610-bc075430bbaf None None] MFA Rules not processed for user `af2b5817e77f4ce7ad0791f5a6ea64af`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:20.225721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2719d762-9bcb-4a70-9610-bc075430bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:20.322513 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1144/2283] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:20.326865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576448-c5f5-46ab-b8e5-89d286b0d26e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:20.459285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576448-c5f5-46ab-b8e5-89d286b0d26e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:20.469669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fdfa7-e506-4ea9-a365-b532cba1204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:20.470418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fdfa7-e506-4ea9-a365-b532cba1204e admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:20.470704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fdfa7-e506-4ea9-a365-b532cba1204e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:20.470981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fdfa7-e506-4ea9-a365-b532cba1204e admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:20.474127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fdfa7-e506-4ea9-a365-b532cba1204e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:20.481672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fdfa7-e506-4ea9-a365-b532cba1204e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:20.545517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1140/2284] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:20 2022] POST /identity/v3/projects => generated 382 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:24:20.554588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719d762-9bcb-4a70-9610-bc075430bbaf None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:20.574924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719d762-9bcb-4a70-9610-bc075430bbaf None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:20.683228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d63a-9826-403a-bee9-e218ceb280ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:20.684155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d63a-9826-403a-bee9-e218ceb280ab admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:20.684542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d63a-9826-403a-bee9-e218ceb280ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:20.686950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d63a-9826-403a-bee9-e218ceb280ab admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:20.686950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d63a-9826-403a-bee9-e218ceb280ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:20.690778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d63a-9826-403a-bee9-e218ceb280ab admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:20.724619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1145/2285] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:20 2022] POST /identity/v3/users => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:20.725843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fdfa7-e506-4ea9-a365-b532cba1204e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:20.739321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fdfa7-e506-4ea9-a365-b532cba1204e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:20.745466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56492c-8080-4c16-a264-7e0218461b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:20.746298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56492c-8080-4c16-a264-7e0218461b79 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:20.746739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56492c-8080-4c16-a264-7e0218461b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:20.747187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56492c-8080-4c16-a264-7e0218461b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:20.748722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56492c-8080-4c16-a264-7e0218461b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:20.751863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56492c-8080-4c16-a264-7e0218461b79 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:20.764790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1141/2286] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:20 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:20.768740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d63a-9826-403a-bee9-e218ceb280ab admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:20.781750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d63a-9826-403a-bee9-e218ceb280ab admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:20.791304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2ef20-0513-4743-99b6-00c4e0e3c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:20.792017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2ef20-0513-4743-99b6-00c4e0e3c84e admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:20.792281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2ef20-0513-4743-99b6-00c4e0e3c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:20.792561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2ef20-0513-4743-99b6-00c4e0e3c84e admin admin] PATH_INFO: `/v3/projects/1a2a531dae684ab5bc8f10ccb408fab9/users/f08d84caf09d48fc892454cf1bb668d2/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:20.817827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2ef20-0513-4743-99b6-00c4e0e3c84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2a531dae684ab5bc8f10ccb408fab9, user_id=f08d84caf09d48fc892454cf1bb668d2, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:20.822316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2ef20-0513-4743-99b6-00c4e0e3c84e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:20.841993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1146/2287] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:20 2022] PUT /identity/v3/projects/1a2a531dae684ab5bc8f10ccb408fab9/users/f08d84caf09d48fc892454cf1bb668d2/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:20.846991 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56492c-8080-4c16-a264-7e0218461b79 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:20.860652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56492c-8080-4c16-a264-7e0218461b79 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:20.884814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:20.936739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822d785-d4aa-410b-b3a6-c668e0b8ac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:20.938393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d785-d4aa-410b-b3a6-c668e0b8ac3c admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:20.938781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d785-d4aa-410b-b3a6-c668e0b8ac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:20.939172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d785-d4aa-410b-b3a6-c668e0b8ac3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:20.940019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822d785-d4aa-410b-b3a6-c668e0b8ac3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:20.942007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822d785-d4aa-410b-b3a6-c668e0b8ac3c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:20.955880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1142/2288] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:20 2022] GET /identity/v3/roles => generated 1427 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:20.959894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2ef20-0513-4743-99b6-00c4e0e3c84e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:20.973398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2ef20-0513-4743-99b6-00c4e0e3c84e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:20.986168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:20.987359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:20.987885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:20.988376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] PATH_INFO: `/v3/projects/1a2a531dae684ab5bc8f10ccb408fab9/users/f08d84caf09d48fc892454cf1bb668d2/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:20.991823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2a531dae684ab5bc8f10ccb408fab9, user_id=f08d84caf09d48fc892454cf1bb668d2, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:20.995917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:21.012392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1147/2289] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:20 2022] PUT /identity/v3/projects/1a2a531dae684ab5bc8f10ccb408fab9/users/f08d84caf09d48fc892454cf1bb668d2/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:21.024200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a337b-bede-4657-9c7d-b5bffbbf09df None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:21.025304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a337b-bede-4657-9c7d-b5bffbbf09df None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:21.025854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a337b-bede-4657-9c7d-b5bffbbf09df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:21.079864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0b6a337b-bede-4657-9c7d-b5bffbbf09df None None] MFA Rules not processed for user `f08d84caf09d48fc892454cf1bb668d2`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:21.081457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0b6a337b-bede-4657-9c7d-b5bffbbf09df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:21.201615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1143/2290] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:21 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:21.214422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:21.216234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:21.266441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:21.309063 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffad32-450f-41d1-8fa5-14ba068af18c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:21.315885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef159be-7440-4395-8a26-5d340eb4e57d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:21.316596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef159be-7440-4395-8a26-5d340eb4e57d service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:21.316948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef159be-7440-4395-8a26-5d340eb4e57d service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:21.317879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef159be-7440-4395-8a26-5d340eb4e57d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:21.331606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef159be-7440-4395-8a26-5d340eb4e57d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:21.333362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef159be-7440-4395-8a26-5d340eb4e57d service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:21.357482 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1148/2291] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:21 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:22.044131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a337b-bede-4657-9c7d-b5bffbbf09df None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:22.162935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a337b-bede-4657-9c7d-b5bffbbf09df None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:22.176761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:22.177583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:22.177931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:22.178412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] PATH_INFO: `/v3/users/af2b5817e77f4ce7ad0791f5a6ea64af` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:22.179815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2b5817e77f4ce7ad0791f5a6ea64af)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:22.182607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:22.279456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2b5817e77f4ce7ad0791f5a6ea64af'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:22.284054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:22.296360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2b5817e77f4ce7ad0791f5a6ea64af'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:22.298214 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2b5817e77f4ce7ad0791f5a6ea64af'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:22.316360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2b5817e77f4ce7ad0791f5a6ea64af'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:22.317519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2b5817e77f4ce7ad0791f5a6ea64af'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:22.360515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1144/2292] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:22 2022] DELETE /identity/v3/users/af2b5817e77f4ce7ad0791f5a6ea64af => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:22.362420 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb51e95-2873-4e98-9f4e-77179ccb266e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:22.363025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb51e95-2873-4e98-9f4e-77179ccb266e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:22.363025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb51e95-2873-4e98-9f4e-77179ccb266e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:22.420067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-bdb51e95-2873-4e98-9f4e-77179ccb266e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:22.423734 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bdb51e95-2873-4e98-9f4e-77179ccb266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:22.537764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1149/2293] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:22.547832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:22.551699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:22.689056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b7018-951c-46f6-ba64-405ab525e655 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:22.693065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1ef6d-6e7e-4f5d-810e-da85fa4a5db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:22.693733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ef6d-6e7e-4f5d-810e-da85fa4a5db4 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:22.694127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ef6d-6e7e-4f5d-810e-da85fa4a5db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:22.694411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ef6d-6e7e-4f5d-810e-da85fa4a5db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:22.708257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1ef6d-6e7e-4f5d-810e-da85fa4a5db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:22.710041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1ef6d-6e7e-4f5d-810e-da85fa4a5db4 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:22.742946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1145/2294] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:23.164748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb51e95-2873-4e98-9f4e-77179ccb266e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:23.166240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bdb51e95-2873-4e98-9f4e-77179ccb266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:23.178995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb51e95-2873-4e98-9f4e-77179ccb266e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:23.184054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:23.184718 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:23.184999 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:23.185289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] PATH_INFO: `/v3/projects/9f46074a3077408cba379bc9e790c660` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:23.186627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f46074a3077408cba379bc9e790c660)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:23.188539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:23.298958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f46074a3077408cba379bc9e790c660', 'request_id': 'req-35269512-8036-47f1-bc0d-b251fcd833a7'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.313151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f46074a3077408cba379bc9e790c660', 'request_id': 'req-35269512-8036-47f1-bc0d-b251fcd833a7'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.328930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f46074a3077408cba379bc9e790c660', 'request_id': 'req-35269512-8036-47f1-bc0d-b251fcd833a7'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.360485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] The token cache is being invalidate because project 9f46074a3077408cba379bc9e790c660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:23.360767 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.362518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.368010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1150/2295] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:23 2022] DELETE /identity/v3/projects/9f46074a3077408cba379bc9e790c660 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:23.369735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1ef6d-6e7e-4f5d-810e-da85fa4a5db4 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:23.387394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1ef6d-6e7e-4f5d-810e-da85fa4a5db4 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:23.492173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:23.493491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:23.493783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:23.494064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] PATH_INFO: `/v3/users/f08d84caf09d48fc892454cf1bb668d2` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:23.495266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08d84caf09d48fc892454cf1bb668d2)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:23.497305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:23.577809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08d84caf09d48fc892454cf1bb668d2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.590200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08d84caf09d48fc892454cf1bb668d2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.594516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08d84caf09d48fc892454cf1bb668d2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.609372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:23.612780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08d84caf09d48fc892454cf1bb668d2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.615774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-170a6435-cd56-4014-a315-ed8dfcea5a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08d84caf09d48fc892454cf1bb668d2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:23.669469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1146/2296] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:23 2022] DELETE /identity/v3/users/f08d84caf09d48fc892454cf1bb668d2 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:24.027774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:24.030416 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:24.046352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35269512-8036-47f1-bc0d-b251fcd833a7 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:24.111229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:24.133783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:24.134616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.135015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.135299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] PATH_INFO: `/v3/projects/1a2a531dae684ab5bc8f10ccb408fab9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.136562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2a531dae684ab5bc8f10ccb408fab9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:24.138804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:24.206657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2a531dae684ab5bc8f10ccb408fab9', 'request_id': 'req-2977c883-7af9-4917-a8e3-df95d809df8b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:24.216007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2a531dae684ab5bc8f10ccb408fab9', 'request_id': 'req-2977c883-7af9-4917-a8e3-df95d809df8b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:24.228680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2a531dae684ab5bc8f10ccb408fab9', 'request_id': 'req-2977c883-7af9-4917-a8e3-df95d809df8b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:24.246078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] The token cache is being invalidate because project 1a2a531dae684ab5bc8f10ccb408fab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:24.246606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:24.247405 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2977c883-7af9-4917-a8e3-df95d809df8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:24.249671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1151/2297] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:24 2022] DELETE /identity/v3/projects/1a2a531dae684ab5bc8f10ccb408fab9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:24.254256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23080-a1fc-45d9-86fb-a47f752b4a42 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.254890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23080-a1fc-45d9-86fb-a47f752b4a42 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.255286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23080-a1fc-45d9-86fb-a47f752b4a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.280788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0ff23080-a1fc-45d9-86fb-a47f752b4a42 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:24.282238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0ff23080-a1fc-45d9-86fb-a47f752b4a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:24.363472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1147/2298] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:24.373283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b9a36-db02-4d82-bf31-7fa1815a7ec2 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.374289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b9a36-db02-4d82-bf31-7fa1815a7ec2 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.374289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b9a36-db02-4d82-bf31-7fa1815a7ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.397153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-820b9a36-db02-4d82-bf31-7fa1815a7ec2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:24.398672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-820b9a36-db02-4d82-bf31-7fa1815a7ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:24.409696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1152/2299] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:24.414094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff23080-a1fc-45d9-86fb-a47f752b4a42 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:24.438709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff23080-a1fc-45d9-86fb-a47f752b4a42 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:24.448785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:24.450356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.451001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.451944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.453476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:24:24.454184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:24.455979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:24.472238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1148/2300] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:24:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:24:24.478342 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b9a36-db02-4d82-bf31-7fa1815a7ec2 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:24.497809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b9a36-db02-4d82-bf31-7fa1815a7ec2 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:24.523553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21baad-89a3-4e9c-9e47-db8627e3c1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:24.524297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21baad-89a3-4e9c-9e47-db8627e3c1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.525555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21baad-89a3-4e9c-9e47-db8627e3c1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.525894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21baad-89a3-4e9c-9e47-db8627e3c1e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.528115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21baad-89a3-4e9c-9e47-db8627e3c1e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:24.548408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21baad-89a3-4e9c-9e47-db8627e3c1e0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:24.630742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:24.638088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1153/2301] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:24 2022] POST /identity/v3/projects => generated 374 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:24.644568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd60951-ed8b-4a14-a5cd-21731602e07a admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:24.775136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9e43-7401-4bd9-a741-8d7e899e994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:24.775136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9e43-7401-4bd9-a741-8d7e899e994b admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.775136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9e43-7401-4bd9-a741-8d7e899e994b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.775136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9e43-7401-4bd9-a741-8d7e899e994b admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.775610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d9e43-7401-4bd9-a741-8d7e899e994b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:24.775610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d9e43-7401-4bd9-a741-8d7e899e994b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:24.799195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1149/2302] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:24 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:24.802856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21baad-89a3-4e9c-9e47-db8627e3c1e0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:24.814888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21baad-89a3-4e9c-9e47-db8627e3c1e0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:24.819848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458836fb-5e8b-4343-8a27-bc2f8baf8a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:24.820589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458836fb-5e8b-4343-8a27-bc2f8baf8a99 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.821021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458836fb-5e8b-4343-8a27-bc2f8baf8a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.821302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458836fb-5e8b-4343-8a27-bc2f8baf8a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.822104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458836fb-5e8b-4343-8a27-bc2f8baf8a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:24.823702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458836fb-5e8b-4343-8a27-bc2f8baf8a99 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:24.834656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1154/2303] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:24 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:24:24.838749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9e43-7401-4bd9-a741-8d7e899e994b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:24.850136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9e43-7401-4bd9-a741-8d7e899e994b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:24.855393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9a258-e072-407c-a1c4-79952ae74132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:24.856104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9a258-e072-407c-a1c4-79952ae74132 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.856451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9a258-e072-407c-a1c4-79952ae74132 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.856798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9a258-e072-407c-a1c4-79952ae74132 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.857633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9a258-e072-407c-a1c4-79952ae74132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:24.859842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9a258-e072-407c-a1c4-79952ae74132 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:24.872099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1150/2304] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:24 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:24:24.876327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458836fb-5e8b-4343-8a27-bc2f8baf8a99 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:24.887068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458836fb-5e8b-4343-8a27-bc2f8baf8a99 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:24.892132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daeeeb1-a2e3-434b-bc86-585609231447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:24.892837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daeeeb1-a2e3-434b-bc86-585609231447 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:24.893118 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daeeeb1-a2e3-434b-bc86-585609231447 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:24.893389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daeeeb1-a2e3-434b-bc86-585609231447 admin admin] PATH_INFO: `/v3/projects/96c597920e9d40b8b8e19a57e999f3ab/users/c41984786cad4de49e6e6ccdf65ab844/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:24.910967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daeeeb1-a2e3-434b-bc86-585609231447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c597920e9d40b8b8e19a57e999f3ab, user_id=c41984786cad4de49e6e6ccdf65ab844, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:24.914408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daeeeb1-a2e3-434b-bc86-585609231447 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:24.932454 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1155/2305] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:24 2022] PUT /identity/v3/projects/96c597920e9d40b8b8e19a57e999f3ab/users/c41984786cad4de49e6e6ccdf65ab844/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:25.003696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15339-8631-4ef1-8724-03836230a74f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.003978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15339-8631-4ef1-8724-03836230a74f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.004249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15339-8631-4ef1-8724-03836230a74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.035256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a4e15339-8631-4ef1-8724-03836230a74f None None] MFA Rules not processed for user `c41984786cad4de49e6e6ccdf65ab844`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:25.036548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a4e15339-8631-4ef1-8724-03836230a74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:25.081136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:25.107851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1151/2306] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:25.112875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daeeeb1-a2e3-434b-bc86-585609231447 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:25.178641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daeeeb1-a2e3-434b-bc86-585609231447 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:25.185658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f33f1-8aa2-4311-a401-7f8255d273b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:25.186570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f33f1-8aa2-4311-a401-7f8255d273b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.186791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f33f1-8aa2-4311-a401-7f8255d273b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.187064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f33f1-8aa2-4311-a401-7f8255d273b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.187740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f33f1-8aa2-4311-a401-7f8255d273b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:25.189650 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f33f1-8aa2-4311-a401-7f8255d273b8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:25.220404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1156/2307] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:25 2022] POST /identity/v3/projects => generated 376 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:24:25.221690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e15339-8631-4ef1-8724-03836230a74f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:25.234534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e15339-8631-4ef1-8724-03836230a74f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:25.291842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e091f-8870-45e9-abeb-7d1e4b38e0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:25.292482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e091f-8870-45e9-abeb-7d1e4b38e0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.292741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e091f-8870-45e9-abeb-7d1e4b38e0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.293008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e091f-8870-45e9-abeb-7d1e4b38e0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.293696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e091f-8870-45e9-abeb-7d1e4b38e0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:25.295824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e091f-8870-45e9-abeb-7d1e4b38e0c6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:25.316077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1152/2308] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:25 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:25.320213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f33f1-8aa2-4311-a401-7f8255d273b8 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:25.327744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:25.332028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f33f1-8aa2-4311-a401-7f8255d273b8 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:25.337819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e4d09-922a-4895-96ee-2b6254f309bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:25.338614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e4d09-922a-4895-96ee-2b6254f309bf admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.339062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e4d09-922a-4895-96ee-2b6254f309bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.339506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e4d09-922a-4895-96ee-2b6254f309bf admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.340200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e4d09-922a-4895-96ee-2b6254f309bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:25.341834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e4d09-922a-4895-96ee-2b6254f309bf admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:25.353053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1157/2309] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:25 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:25.357290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e091f-8870-45e9-abeb-7d1e4b38e0c6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:25.368123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e091f-8870-45e9-abeb-7d1e4b38e0c6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:25.372834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f3545-45d4-4904-a69c-a76654c16c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:25.373528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f3545-45d4-4904-a69c-a76654c16c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.373812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f3545-45d4-4904-a69c-a76654c16c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.374087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f3545-45d4-4904-a69c-a76654c16c0a admin admin] PATH_INFO: `/v3/projects/62646c89a1434329baf0ba2a42cc4c5a/users/5716b608b21d43eea49b9e5c8945fe5b/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.388601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f3545-45d4-4904-a69c-a76654c16c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=62646c89a1434329baf0ba2a42cc4c5a, user_id=5716b608b21d43eea49b9e5c8945fe5b, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:25.391863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f3545-45d4-4904-a69c-a76654c16c0a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:25.404356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1153/2310] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:25 2022] PUT /identity/v3/projects/62646c89a1434329baf0ba2a42cc4c5a/users/5716b608b21d43eea49b9e5c8945fe5b/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:25.408912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e4d09-922a-4895-96ee-2b6254f309bf admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:25.418840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e4d09-922a-4895-96ee-2b6254f309bf admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:25.475694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f0010-1d23-4055-8c71-f659765332ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:25.476337 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f0010-1d23-4055-8c71-f659765332ae admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.476619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f0010-1d23-4055-8c71-f659765332ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.476891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f0010-1d23-4055-8c71-f659765332ae admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.477615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f0010-1d23-4055-8c71-f659765332ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:25.479237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f0010-1d23-4055-8c71-f659765332ae admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:25.488801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1158/2311] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:25 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:25.493227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f3545-45d4-4904-a69c-a76654c16c0a admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:25.504652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f3545-45d4-4904-a69c-a76654c16c0a admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:25.509341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:25.510046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.510328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.510599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] PATH_INFO: `/v3/projects/62646c89a1434329baf0ba2a42cc4c5a/users/5716b608b21d43eea49b9e5c8945fe5b/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.512576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62646c89a1434329baf0ba2a42cc4c5a, user_id=5716b608b21d43eea49b9e5c8945fe5b, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:25.515872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:25.526093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1154/2312] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:25 2022] PUT /identity/v3/projects/62646c89a1434329baf0ba2a42cc4c5a/users/5716b608b21d43eea49b9e5c8945fe5b/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:25.536215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628112a-e54c-47a2-bd57-47b16d8b856d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.536501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628112a-e54c-47a2-bd57-47b16d8b856d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.536800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628112a-e54c-47a2-bd57-47b16d8b856d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.564754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f628112a-e54c-47a2-bd57-47b16d8b856d None None] MFA Rules not processed for user `5716b608b21d43eea49b9e5c8945fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:25.565980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f628112a-e54c-47a2-bd57-47b16d8b856d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:25.646728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1159/2313] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:25.657766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:25.659767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:25.723615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc4eee-2fa7-4e56-9b24-d08ae4e8b874 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:25.727248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:25.727862 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:25.728639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:25.728639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:25.775130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:25.786933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:25.788374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:25.798759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1155/2314] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:24:26.348918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759a36-dc7b-4e1f-8b69-09315afd9473 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:26.349214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759a36-dc7b-4e1f-8b69-09315afd9473 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:26.349529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759a36-dc7b-4e1f-8b69-09315afd9473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:26.370528 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f7759a36-dc7b-4e1f-8b69-09315afd9473 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:26.371715 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f7759a36-dc7b-4e1f-8b69-09315afd9473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:26.425855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1160/2315] 10.222.0.109 () {58 vars in 972 bytes} [Thu Aug 4 22:24:26 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:26.438726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:26.440105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:26.509021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744231b-086f-4a26-8311-31a881d0bade service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:26.514108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:26.514722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:26.515001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:26.515815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:26.524375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:26.526011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:26.534103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1156/2316] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:24:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:24:28.581771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759a36-dc7b-4e1f-8b69-09315afd9473 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:28.635561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:28.690966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759a36-dc7b-4e1f-8b69-09315afd9473 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:28.696161 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:28.696813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:28.697123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:28.697401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] PATH_INFO: `/v3/users/c41984786cad4de49e6e6ccdf65ab844` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:28.698609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41984786cad4de49e6e6ccdf65ab844)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:28.700687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:28.783854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41984786cad4de49e6e6ccdf65ab844'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.795800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41984786cad4de49e6e6ccdf65ab844'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.802827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:28.804593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:28.817565 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41984786cad4de49e6e6ccdf65ab844'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.819288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41984786cad4de49e6e6ccdf65ab844'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.825223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80f283-971a-42d9-81e6-d634e354e15d service swift] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:28.837240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:28.837947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:28.838232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:28.838502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] PATH_INFO: `/v3/users/9542521c42b34769a236eb6f268c5358` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:28.839192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5b4bcfea-8c67-479f-8617-37a464ba8b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41984786cad4de49e6e6ccdf65ab844'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.839694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9542521c42b34769a236eb6f268c5358)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:28.842161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:28.847510 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1161/2317] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:28 2022] DELETE /identity/v3/users/c41984786cad4de49e6e6ccdf65ab844 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:28.851689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5222c7d-0fe1-4304-a723-e388ba4786fd None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:28.851979 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5222c7d-0fe1-4304-a723-e388ba4786fd None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:28.852259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5222c7d-0fe1-4304-a723-e388ba4786fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:28.892880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a5222c7d-0fe1-4304-a723-e388ba4786fd None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:28.894138 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a5222c7d-0fe1-4304-a723-e388ba4786fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:28.925820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9542521c42b34769a236eb6f268c5358'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.949636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9542521c42b34769a236eb6f268c5358'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.950291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9542521c42b34769a236eb6f268c5358'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.969954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9542521c42b34769a236eb6f268c5358'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.970813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9542521c42b34769a236eb6f268c5358'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:28.997751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1157/2318] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:28 2022] DELETE /identity/v3/users/9542521c42b34769a236eb6f268c5358 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:29.006978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1162/2319] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:29.026674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:29.031092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:29.134185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595acdd5-9f75-4770-a5e4-8cf6c25a1607 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:29.139934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d5580-f916-4d03-850a-d9d696d8dc4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:29.141189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d5580-f916-4d03-850a-d9d696d8dc4b service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:29.141521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d5580-f916-4d03-850a-d9d696d8dc4b service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:29.141803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d5580-f916-4d03-850a-d9d696d8dc4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:29.148640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:29.220127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d5580-f916-4d03-850a-d9d696d8dc4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:29.221812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d5580-f916-4d03-850a-d9d696d8dc4b service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:29.235224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1158/2320] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:24:29.319108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5222c7d-0fe1-4304-a723-e388ba4786fd None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:29.324963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a5222c7d-0fe1-4304-a723-e388ba4786fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:29.360572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5222c7d-0fe1-4304-a723-e388ba4786fd None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:29.377634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:29.378527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:29.378913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:29.379291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] PATH_INFO: `/v3/projects/9017e927b016427aa84b863e0e2a86a4` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:29.381160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9017e927b016427aa84b863e0e2a86a4)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:29.384182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:29.454423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:29.489058 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9017e927b016427aa84b863e0e2a86a4', 'request_id': 'req-432b19dc-445b-4146-b02b-b711182e1125'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:29.506689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9017e927b016427aa84b863e0e2a86a4', 'request_id': 'req-432b19dc-445b-4146-b02b-b711182e1125'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:29.517890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9017e927b016427aa84b863e0e2a86a4', 'request_id': 'req-432b19dc-445b-4146-b02b-b711182e1125'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:29.534424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] The token cache is being invalidate because project 9017e927b016427aa84b863e0e2a86a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:29.534866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:29.535640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-432b19dc-445b-4146-b02b-b711182e1125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:29.541201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1163/2321] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:29 2022] DELETE /identity/v3/projects/9017e927b016427aa84b863e0e2a86a4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:29.542585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced46210-b755-4ca8-b436-c52a4b61b83c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:29.542869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced46210-b755-4ca8-b436-c52a4b61b83c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:29.543686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced46210-b755-4ca8-b436-c52a4b61b83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:29.573264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ced46210-b755-4ca8-b436-c52a4b61b83c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:29.577025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ced46210-b755-4ca8-b436-c52a4b61b83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:29.733434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1159/2322] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:29.734249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034b049-956a-49e0-b92b-1cc2c72a0d67 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:29.734249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034b049-956a-49e0-b92b-1cc2c72a0d67 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:29.734249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034b049-956a-49e0-b92b-1cc2c72a0d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:29.774236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-d034b049-956a-49e0-b92b-1cc2c72a0d67 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:29.775578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d034b049-956a-49e0-b92b-1cc2c72a0d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:29.791819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1164/2323] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:29.796581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced46210-b755-4ca8-b436-c52a4b61b83c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:29.808372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced46210-b755-4ca8-b436-c52a4b61b83c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:29.822124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:29.823332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:29.823807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:29.824393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:29.825294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:24:29.826030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:29.828213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:29.855860 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1160/2324] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:24:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:24:29.860126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034b049-956a-49e0-b92b-1cc2c72a0d67 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:29.875110 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034b049-956a-49e0-b92b-1cc2c72a0d67 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:29.880026 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:29.880623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:29.880880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:29.881314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:29.882198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:29.884237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:29.931429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1165/2325] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:24:29 2022] POST /identity/v3/projects => generated 392 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:29.936440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:29.947471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbbdf7-3415-4549-85c2-a3a665896afb admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:30.015808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b506c5-5598-4b1d-9cde-e7648a02f006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:30.016441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b506c5-5598-4b1d-9cde-e7648a02f006 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.017050 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b506c5-5598-4b1d-9cde-e7648a02f006 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.017339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b506c5-5598-4b1d-9cde-e7648a02f006 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.018043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b506c5-5598-4b1d-9cde-e7648a02f006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:30.020523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b506c5-5598-4b1d-9cde-e7648a02f006 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:30.044135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:30.046806 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:30.057133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1161/2326] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:24:29 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:24:30.063175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b506c5-5598-4b1d-9cde-e7648a02f006 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:30.064285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7873f-fa25-4448-a5c4-bcdbfe676383 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:30.069176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:30.069947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.070648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.071139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] PATH_INFO: `/v3/projects/96c597920e9d40b8b8e19a57e999f3ab` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.072761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c597920e9d40b8b8e19a57e999f3ab)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:30.074985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:30.081503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b506c5-5598-4b1d-9cde-e7648a02f006 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:30.088060 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30f8e-dc78-4e50-9928-73535c53b56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:30.088911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30f8e-dc78-4e50-9928-73535c53b56e admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.089335 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30f8e-dc78-4e50-9928-73535c53b56e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.090093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30f8e-dc78-4e50-9928-73535c53b56e admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.090917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e30f8e-dc78-4e50-9928-73535c53b56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:30.092672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e30f8e-dc78-4e50-9928-73535c53b56e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:30.126126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1162/2327] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:30 2022] GET /identity/v3/roles => generated 1427 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:24:30.131343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30f8e-dc78-4e50-9928-73535c53b56e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:30.144629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30f8e-dc78-4e50-9928-73535c53b56e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:30.150936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:30.164171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c597920e9d40b8b8e19a57e999f3ab', 'request_id': 'req-75595527-ee02-4d70-9a5f-d485f1abdf0b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.173694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c597920e9d40b8b8e19a57e999f3ab', 'request_id': 'req-75595527-ee02-4d70-9a5f-d485f1abdf0b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.186107 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c597920e9d40b8b8e19a57e999f3ab', 'request_id': 'req-75595527-ee02-4d70-9a5f-d485f1abdf0b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.210809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] The token cache is being invalidate because project 96c597920e9d40b8b8e19a57e999f3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:30.211551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.212631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.215218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1166/2328] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:30 2022] DELETE /identity/v3/projects/96c597920e9d40b8b8e19a57e999f3ab => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:24:30.219310 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:30.221154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:30.238271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:30.239396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.239854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.240579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.242256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:30.244078 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:30.257980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1163/2329] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:24:30 2022] GET /identity/v3/roles => generated 1427 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:24:30.265677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:30.267790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:30.285816 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236bbf-073e-4bdf-aad2-f9c82cf245cb admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:30.292343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72560306-1096-47f3-8d15-2a1292103beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:30.293884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72560306-1096-47f3-8d15-2a1292103beb admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.294617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72560306-1096-47f3-8d15-2a1292103beb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.294885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72560306-1096-47f3-8d15-2a1292103beb admin admin] PATH_INFO: `/v3/projects/f9cc96b9fd874e3ebc2325c90f68b490/users/e248a64925c04d07b3c2f93ad2c5d05e/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.311439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72560306-1096-47f3-8d15-2a1292103beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cc96b9fd874e3ebc2325c90f68b490, user_id=e248a64925c04d07b3c2f93ad2c5d05e, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:30.314873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72560306-1096-47f3-8d15-2a1292103beb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:30.316487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595527-ee02-4d70-9a5f-d485f1abdf0b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:30.321579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:30.322235 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.322566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.323390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] PATH_INFO: `/v3/users/5716b608b21d43eea49b9e5c8945fe5b` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.325016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=5716b608b21d43eea49b9e5c8945fe5b)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:30.327172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:30.341656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1164/2330] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:24:30 2022] PUT /identity/v3/projects/f9cc96b9fd874e3ebc2325c90f68b490/users/e248a64925c04d07b3c2f93ad2c5d05e/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:30.347808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a499dd-6389-457f-9cb4-9a08f415a49d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.348094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a499dd-6389-457f-9cb4-9a08f415a49d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.348372 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a499dd-6389-457f-9cb4-9a08f415a49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.352540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:30.381384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-68a499dd-6389-457f-9cb4-9a08f415a49d None None] MFA Rules not processed for user `e248a64925c04d07b3c2f93ad2c5d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:30.381601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5716b608b21d43eea49b9e5c8945fe5b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.382631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-68a499dd-6389-457f-9cb4-9a08f415a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:30.388930 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5716b608b21d43eea49b9e5c8945fe5b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.398679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5716b608b21d43eea49b9e5c8945fe5b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.399281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5716b608b21d43eea49b9e5c8945fe5b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.412276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5716b608b21d43eea49b9e5c8945fe5b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.418357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1167/2331] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:30 2022] DELETE /identity/v3/users/5716b608b21d43eea49b9e5c8945fe5b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:30.449261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1165/2332] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:30 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:24:30.461007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:30.462339 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:30.524343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09b156-cab7-45ff-836a-5f99f3349fde admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:30.528025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:30.528764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.529054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.529314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.592182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:30.594394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:30.603385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1168/2333] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:30 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:30.698038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a499dd-6389-457f-9cb4-9a08f415a49d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:30.699463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-68a499dd-6389-457f-9cb4-9a08f415a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:30.787119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a499dd-6389-457f-9cb4-9a08f415a49d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:30.803138 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:30.861977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:30.862927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:30.863065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:30.863404 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] PATH_INFO: `/v3/projects/62646c89a1434329baf0ba2a42cc4c5a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:30.865365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62646c89a1434329baf0ba2a42cc4c5a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:30.867621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:30.928828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62646c89a1434329baf0ba2a42cc4c5a', 'request_id': 'req-91827f37-3974-4028-a17e-e1f25d123343'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.936529 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62646c89a1434329baf0ba2a42cc4c5a', 'request_id': 'req-91827f37-3974-4028-a17e-e1f25d123343'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.942615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62646c89a1434329baf0ba2a42cc4c5a', 'request_id': 'req-91827f37-3974-4028-a17e-e1f25d123343'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.955408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] The token cache is being invalidate because project 62646c89a1434329baf0ba2a42cc4c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:30.955732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.956390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-91827f37-3974-4028-a17e-e1f25d123343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:30.957951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1166/2334] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:30 2022] DELETE /identity/v3/projects/62646c89a1434329baf0ba2a42cc4c5a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:38.529202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:38.530663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:38.548073 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0e127-52e0-48bd-8c54-8560145aca03 service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:38.597033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:38.618272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:38.619687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:38.619687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:38.620111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] PATH_INFO: `/v3/users/755df1ae00c6484780dcfba580935d28` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:38.621926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=755df1ae00c6484780dcfba580935d28)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:38.624238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:38.685116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755df1ae00c6484780dcfba580935d28'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:38.694498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755df1ae00c6484780dcfba580935d28'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:38.707608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755df1ae00c6484780dcfba580935d28'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:38.708331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755df1ae00c6484780dcfba580935d28'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:38.728311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755df1ae00c6484780dcfba580935d28'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:38.736854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1169/2335] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:38 2022] DELETE /identity/v3/users/755df1ae00c6484780dcfba580935d28 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:38.741427 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c2056-43a5-4c9f-98f8-6fe6fb11c705 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:38.741917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c2056-43a5-4c9f-98f8-6fe6fb11c705 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:38.742021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c2056-43a5-4c9f-98f8-6fe6fb11c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:38.763507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9f8c2056-43a5-4c9f-98f8-6fe6fb11c705 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:38.764736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f8c2056-43a5-4c9f-98f8-6fe6fb11c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:38.835456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1167/2336] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:38.845456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:38.846905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:38.906905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b313a-4731-413a-ba28-5afd53bb2cac admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:38.911092 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05a97b-041d-472e-97e5-b7f594e5f065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:38.911841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05a97b-041d-472e-97e5-b7f594e5f065 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:38.912193 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05a97b-041d-472e-97e5-b7f594e5f065 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:38.912493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05a97b-041d-472e-97e5-b7f594e5f065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:38.922051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05a97b-041d-472e-97e5-b7f594e5f065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:38.923689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05a97b-041d-472e-97e5-b7f594e5f065 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:38.933131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1170/2337] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:39.202589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c2056-43a5-4c9f-98f8-6fe6fb11c705 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:39.204668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f8c2056-43a5-4c9f-98f8-6fe6fb11c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:39.217502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c2056-43a5-4c9f-98f8-6fe6fb11c705 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:39.223293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:39.223956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:39.224234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:39.224676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] PATH_INFO: `/v3/projects/467075e54e354e38b220629fbf5df535` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:39.225957 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=467075e54e354e38b220629fbf5df535)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:39.227889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:39.275761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:39.283913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467075e54e354e38b220629fbf5df535', 'request_id': 'req-7111510a-5e0a-4711-a21b-16c188e49f1d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:39.295687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467075e54e354e38b220629fbf5df535', 'request_id': 'req-7111510a-5e0a-4711-a21b-16c188e49f1d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:39.301865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467075e54e354e38b220629fbf5df535', 'request_id': 'req-7111510a-5e0a-4711-a21b-16c188e49f1d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:39.313905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] The token cache is being invalidate because project 467075e54e354e38b220629fbf5df535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:39.314194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:39.314847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:39.316373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1168/2338] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:39 2022] DELETE /identity/v3/projects/467075e54e354e38b220629fbf5df535 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:45.414274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6890-085b-47c8-902d-78e205c7471c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:45.414779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6890-085b-47c8-902d-78e205c7471c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:45.414872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6890-085b-47c8-902d-78e205c7471c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:45.434241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9a9a6890-085b-47c8-902d-78e205c7471c None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:45.435460 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9a9a6890-085b-47c8-902d-78e205c7471c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:45.495667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1171/2339] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:24:45 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:45.508892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:45.510446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:45.585876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111510a-5e0a-4711-a21b-16c188e49f1d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:45.591468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2986228-d9ec-4525-a63c-1aceb2f7061a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:45.592170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2986228-d9ec-4525-a63c-1aceb2f7061a service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:45.592433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2986228-d9ec-4525-a63c-1aceb2f7061a service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:45.592713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2986228-d9ec-4525-a63c-1aceb2f7061a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:45.601990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2986228-d9ec-4525-a63c-1aceb2f7061a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:45.603519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2986228-d9ec-4525-a63c-1aceb2f7061a service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:45.620307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1169/2340] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:24:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:24:46.238780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d51f7-77eb-4748-b8b8-c4e0749882e2 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:46.239068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d51f7-77eb-4748-b8b8-c4e0749882e2 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:46.239601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d51f7-77eb-4748-b8b8-c4e0749882e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:46.260206 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-0c1d51f7-77eb-4748-b8b8-c4e0749882e2 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:46.261421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0c1d51f7-77eb-4748-b8b8-c4e0749882e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:46.269286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1172/2341] 10.222.0.109 () {58 vars in 972 bytes} [Thu Aug 4 22:24:46 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:46.281509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2986228-d9ec-4525-a63c-1aceb2f7061a service swift] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:46.292125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2986228-d9ec-4525-a63c-1aceb2f7061a service swift] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:46.297455 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79920b1-5d11-4a1c-849e-c18e137484dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:46.298087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79920b1-5d11-4a1c-849e-c18e137484dc service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:46.298365 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79920b1-5d11-4a1c-849e-c18e137484dc service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:46.298718 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79920b1-5d11-4a1c-849e-c18e137484dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:46.307796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79920b1-5d11-4a1c-849e-c18e137484dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:46.309503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79920b1-5d11-4a1c-849e-c18e137484dc service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:46.319605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1170/2342] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:24:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:24:46.536704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408265d-10ed-462b-ade7-167e11b2015a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:46.537171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408265d-10ed-462b-ade7-167e11b2015a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:46.537283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408265d-10ed-462b-ade7-167e11b2015a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:46.556380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8408265d-10ed-462b-ade7-167e11b2015a None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:46.557595 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8408265d-10ed-462b-ade7-167e11b2015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:46.565012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1173/2343] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:24:46 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:46.576654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79920b1-5d11-4a1c-849e-c18e137484dc service swift] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:46.587687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79920b1-5d11-4a1c-849e-c18e137484dc service swift] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:46.593963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13abff-377b-4476-9995-c7a1cdc91e64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:46.594566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13abff-377b-4476-9995-c7a1cdc91e64 service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:46.594841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13abff-377b-4476-9995-c7a1cdc91e64 service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:46.595105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13abff-377b-4476-9995-c7a1cdc91e64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:46.603165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13abff-377b-4476-9995-c7a1cdc91e64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:46.604735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13abff-377b-4476-9995-c7a1cdc91e64 service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:46.612869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1171/2344] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:24:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:24:46.927678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22346e-7da2-4ad4-9eb7-a4466a1119b0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:46.927969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22346e-7da2-4ad4-9eb7-a4466a1119b0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:46.928242 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22346e-7da2-4ad4-9eb7-a4466a1119b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:46.938271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:46.950860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-cc22346e-7da2-4ad4-9eb7-a4466a1119b0 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:46.952042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cc22346e-7da2-4ad4-9eb7-a4466a1119b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:46.959932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1174/2345] 10.222.0.109 () {58 vars in 972 bytes} [Thu Aug 4 22:24:46 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:46.972075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13abff-377b-4476-9995-c7a1cdc91e64 service swift] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:46.982745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13abff-377b-4476-9995-c7a1cdc91e64 service swift] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:46.988182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab7252-170b-49df-ab74-151efb166dde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:46.988787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab7252-170b-49df-ab74-151efb166dde service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:46.989065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab7252-170b-49df-ab74-151efb166dde service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:46.989621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab7252-170b-49df-ab74-151efb166dde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:46.997653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab7252-170b-49df-ab74-151efb166dde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:46.999236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab7252-170b-49df-ab74-151efb166dde service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:47.007623 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1172/2346] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:24:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:24:52.269875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22346e-7da2-4ad4-9eb7-a4466a1119b0 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:52.277464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cc22346e-7da2-4ad4-9eb7-a4466a1119b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:52.346759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22346e-7da2-4ad4-9eb7-a4466a1119b0 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:52.352475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:52.353144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:52.353444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:52.353729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] PATH_INFO: `/v3/users/35858dc56fdf4661a8aaffad520fc96f` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:52.354951 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=35858dc56fdf4661a8aaffad520fc96f)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:52.356875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:52.412223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35858dc56fdf4661a8aaffad520fc96f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.419618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35858dc56fdf4661a8aaffad520fc96f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.431471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35858dc56fdf4661a8aaffad520fc96f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.432286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35858dc56fdf4661a8aaffad520fc96f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.447253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35858dc56fdf4661a8aaffad520fc96f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.456668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1175/2347] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:52 2022] DELETE /identity/v3/users/35858dc56fdf4661a8aaffad520fc96f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:52.462663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbc026-1684-4b0a-9831-47736ff01def None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:52.463129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbc026-1684-4b0a-9831-47736ff01def None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:52.463243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbc026-1684-4b0a-9831-47736ff01def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:52.485383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-d8bbc026-1684-4b0a-9831-47736ff01def None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:52.486657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d8bbc026-1684-4b0a-9831-47736ff01def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:52.549830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1173/2348] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:52.559410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:52.560962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:52.621590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180496f6-d2a5-4b17-84f7-e381d54e93ab admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:52.626080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:52.626701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:52.626984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:52.627260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:52.636793 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:52.638258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:52.650690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1176/2349] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:52.831254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbc026-1684-4b0a-9831-47736ff01def None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:52.832667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d8bbc026-1684-4b0a-9831-47736ff01def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:52.846830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbc026-1684-4b0a-9831-47736ff01def None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:52.851533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:52.852193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:52.852640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:52.852916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] PATH_INFO: `/v3/projects/8cfbac5e5eab4e719af887179d6bcea8` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:52.854223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cfbac5e5eab4e719af887179d6bcea8)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:52.856070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:52.875142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:52.907926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cfbac5e5eab4e719af887179d6bcea8', 'request_id': 'req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.916518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cfbac5e5eab4e719af887179d6bcea8', 'request_id': 'req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.922190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cfbac5e5eab4e719af887179d6bcea8', 'request_id': 'req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.933392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] The token cache is being invalidate because project 8cfbac5e5eab4e719af887179d6bcea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:52.933713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.934381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-28edb902-4df6-4ebe-82ac-f2ad8d7354ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:52.935926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1174/2350] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:52 2022] DELETE /identity/v3/projects/8cfbac5e5eab4e719af887179d6bcea8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:57.235628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:57.237069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:57.251469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69d1b-4b3e-4e6f-9178-aabd380b6b95 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:57.267938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:24:57.309593 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:57.310248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:57.310530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:57.310805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] PATH_INFO: `/v3/users/e248a64925c04d07b3c2f93ad2c5d05e` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:57.312660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e248a64925c04d07b3c2f93ad2c5d05e)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:57.314630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:57.361202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e248a64925c04d07b3c2f93ad2c5d05e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.368592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e248a64925c04d07b3c2f93ad2c5d05e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.375377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e248a64925c04d07b3c2f93ad2c5d05e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.376105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e248a64925c04d07b3c2f93ad2c5d05e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.387981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e248a64925c04d07b3c2f93ad2c5d05e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.394536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1177/2351] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:24:57 2022] DELETE /identity/v3/users/e248a64925c04d07b3c2f93ad2c5d05e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:24:57.398944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b4f7-760b-43d3-89f0-52165cc65506 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:57.399475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b4f7-760b-43d3-89f0-52165cc65506 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:57.399475 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b4f7-760b-43d3-89f0-52165cc65506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:57.419567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e372b4f7-760b-43d3-89f0-52165cc65506 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:24:57.421663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e372b4f7-760b-43d3-89f0-52165cc65506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:57.481804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1175/2352] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:24:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:24:57.492603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:57.494080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:57.556293 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe21288-5ada-4ac1-b3a8-85328d11794b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:57.559889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0381a-94af-47c1-b238-71d8fdce6bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:57.560497 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0381a-94af-47c1-b238-71d8fdce6bdd service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:57.560771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0381a-94af-47c1-b238-71d8fdce6bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:57.561037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0381a-94af-47c1-b238-71d8fdce6bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:57.569840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0381a-94af-47c1-b238-71d8fdce6bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:57.571317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0381a-94af-47c1-b238-71d8fdce6bdd service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:57.581872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1178/2353] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:24:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:24:57.859325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372b4f7-760b-43d3-89f0-52165cc65506 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:24:57.861573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e372b4f7-760b-43d3-89f0-52165cc65506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:24:57.876018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372b4f7-760b-43d3-89f0-52165cc65506 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:24:57.882313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:24:57.883114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:24:57.883497 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:24:57.884277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] PATH_INFO: `/v3/projects/f9cc96b9fd874e3ebc2325c90f68b490` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:24:57.885451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9cc96b9fd874e3ebc2325c90f68b490)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:24:57.887488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:24:57.942755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cc96b9fd874e3ebc2325c90f68b490', 'request_id': 'req-1cad4851-6e36-4914-ad3a-bae63af07bba'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.950741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cc96b9fd874e3ebc2325c90f68b490', 'request_id': 'req-1cad4851-6e36-4914-ad3a-bae63af07bba'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.956652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cc96b9fd874e3ebc2325c90f68b490', 'request_id': 'req-1cad4851-6e36-4914-ad3a-bae63af07bba'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.969061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] The token cache is being invalidate because project f9cc96b9fd874e3ebc2325c90f68b490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:24:57.969530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.970203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:24:57.973500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1176/2354] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:24:57 2022] DELETE /identity/v3/projects/f9cc96b9fd874e3ebc2325c90f68b490 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:25:07.577809 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=70,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:25:07.965220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:25:10.881730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0381a-94af-47c1-b238-71d8fdce6bdd service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:10.890532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0381a-94af-47c1-b238-71d8fdce6bdd service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:10.941042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:10.941819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:10.942094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:10.942361 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:10.944119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:11.014287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:11.015862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:11.027338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1179/2355] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:13.402432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:13.403974 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:13.417657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad4851-6e36-4914-ad3a-bae63af07bba admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:13.422869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d050f3-f26d-4e9b-8369-60eeac944956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:13.423501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d050f3-f26d-4e9b-8369-60eeac944956 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:13.423818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d050f3-f26d-4e9b-8369-60eeac944956 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:13.424037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d050f3-f26d-4e9b-8369-60eeac944956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:13.437006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-83d050f3-f26d-4e9b-8369-60eeac944956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:13.451216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d050f3-f26d-4e9b-8369-60eeac944956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:13.453343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d050f3-f26d-4e9b-8369-60eeac944956 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:13.464917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1177/2356] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:13 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:16.976481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:16.978162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:16.992335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d2ca4-a3c7-4668-8cb2-381c3bfe4564 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:16.997920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:16.999117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:16.999462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:16.999758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:17.001864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:17.069613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:17.071290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:17.081662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1180/2357] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:18.033861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cef11c-6b5b-446d-82ee-40f66f84131f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:18.034388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cef11c-6b5b-446d-82ee-40f66f84131f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:18.034506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cef11c-6b5b-446d-82ee-40f66f84131f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:18.053745 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-d7cef11c-6b5b-446d-82ee-40f66f84131f None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:25:18.054863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d7cef11c-6b5b-446d-82ee-40f66f84131f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:18.109972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1178/2358] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:25:18 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:25:18.132003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:18.133548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:18.179544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:25:18.206125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cc153-c864-4bec-8a41-d0d8cfa7a124 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:18.211210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:18.211826 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:18.212107 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:18.212381 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:18.221205 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:18.222833 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:18.230914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1181/2359] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:25:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:25:24.025449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a0903-de4d-475d-9187-0e44b8283d62 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:24.027345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a0903-de4d-475d-9187-0e44b8283d62 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:24.028204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a0903-de4d-475d-9187-0e44b8283d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:24.055824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b54a0903-de4d-475d-9187-0e44b8283d62 None None] MFA Rules not processed for user `8c50160e51b54cde9a3f617d71fbf1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:25:24.062603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b54a0903-de4d-475d-9187-0e44b8283d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:24.167827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1179/2360] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:25:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:25:24.982983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:24.984615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:25.066427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f8a6c-cc8d-47b6-890d-cf8150dd81f3 service swift] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:25.081558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:25.083001 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:25.083529 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:25.084025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:25.086303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:25.110847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:25.113352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:25.125638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1182/2361] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:24 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:30.873389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a0903-de4d-475d-9187-0e44b8283d62 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:30.875015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b54a0903-de4d-475d-9187-0e44b8283d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:30.891442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a0903-de4d-475d-9187-0e44b8283d62 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:30.896914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b3801-0807-48fc-93c9-dd2ea51bc55f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:30.897552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b3801-0807-48fc-93c9-dd2ea51bc55f service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:30.897833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b3801-0807-48fc-93c9-dd2ea51bc55f service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:30.898103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b3801-0807-48fc-93c9-dd2ea51bc55f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:30.899777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8e6b3801-0807-48fc-93c9-dd2ea51bc55f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:30.910065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b3801-0807-48fc-93c9-dd2ea51bc55f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:30.911828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b3801-0807-48fc-93c9-dd2ea51bc55f service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:30.922077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1180/2362] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:35.120456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=46 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:25:40.917256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=62 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:25:48.973938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:48.983088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30353293-cff9-4a0f-aaf4-a25d47e1fd94 service placement] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:48.986925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db10da-715a-40f7-adbc-0d91773f0ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:48.987518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db10da-715a-40f7-adbc-0d91773f0ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:48.987791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db10da-715a-40f7-adbc-0d91773f0ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:48.988055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db10da-715a-40f7-adbc-0d91773f0ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:48.989959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-24db10da-715a-40f7-adbc-0d91773f0ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:48.999560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db10da-715a-40f7-adbc-0d91773f0ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:49.001623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db10da-715a-40f7-adbc-0d91773f0ad1 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:49.009513 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1183/2363] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:48 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:49.147591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827dd8-b25a-45a4-9c09-9dc416b53994 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.148158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827dd8-b25a-45a4-9c09-9dc416b53994 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.148158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827dd8-b25a-45a4-9c09-9dc416b53994 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.149530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1181/2364] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:25:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:25:49.157935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9a8f8-7671-4925-8d23-e6f550f573f7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.158265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9a8f8-7671-4925-8d23-e6f550f573f7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.158513 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9a8f8-7671-4925-8d23-e6f550f573f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.183404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-38e9a8f8-7671-4925-8d23-e6f550f573f7 None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:25:49.184632 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-38e9a8f8-7671-4925-8d23-e6f550f573f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:49.196176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1184/2365] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:25:49 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:25:49.209739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827dd8-b25a-45a4-9c09-9dc416b53994 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:49.211439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-c2827dd8-b25a-45a4-9c09-9dc416b53994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:49.272061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827dd8-b25a-45a4-9c09-9dc416b53994 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:49.275625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e28ef-b0d6-410b-b142-246b174c5737 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33183e28de8f487f82f92f0151e7c73c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:49.276237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e28ef-b0d6-410b-b142-246b174c5737 service glance] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.276516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e28ef-b0d6-410b-b142-246b174c5737 service glance] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.276786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e28ef-b0d6-410b-b142-246b174c5737 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.287499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e28ef-b0d6-410b-b142-246b174c5737 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:49.288946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e28ef-b0d6-410b-b142-246b174c5737 service glance] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:49.300315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1182/2366] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:49 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:49.368848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84d37b-80e0-4cb7-a72c-c942733fe38f None None] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.369162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84d37b-80e0-4cb7-a72c-c942733fe38f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.369477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84d37b-80e0-4cb7-a72c-c942733fe38f None None] PATH_INFO: `/` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.370899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1185/2367] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:25:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:25:49.376522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c38ec-51b5-4c56-8841-e352172917de None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.376920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c38ec-51b5-4c56-8841-e352172917de None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.377207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c38ec-51b5-4c56-8841-e352172917de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.396674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-4e6c38ec-51b5-4c56-8841-e352172917de None None] MFA Rules not processed for user `5455c117e349480ba8b8c020b767bec5`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:25:49.397918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4e6c38ec-51b5-4c56-8841-e352172917de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:49.404359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1183/2368] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:25:49 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:25:49.416614 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84d37b-80e0-4cb7-a72c-c942733fe38f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:49.418390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bd84d37b-80e0-4cb7-a72c-c942733fe38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:49.432195 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84d37b-80e0-4cb7-a72c-c942733fe38f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:49.435786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a7678-1597-4757-bc94-b6b0e50aeac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:49.436384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7678-1597-4757-bc94-b6b0e50aeac5 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.436659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7678-1597-4757-bc94-b6b0e50aeac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.436927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7678-1597-4757-bc94-b6b0e50aeac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.445056 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a7678-1597-4757-bc94-b6b0e50aeac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:49.446454 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a7678-1597-4757-bc94-b6b0e50aeac5 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:49.454322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1186/2369] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:49 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:49.538627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe6a29-2ed9-48e9-a75e-eba9c539ea83 None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.538924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe6a29-2ed9-48e9-a75e-eba9c539ea83 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.539224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe6a29-2ed9-48e9-a75e-eba9c539ea83 None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.540574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1184/2370] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:25:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:25:49.546045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e3c4-9a80-4f61-ac4a-9f236a413f9a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.546331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e3c4-9a80-4f61-ac4a-9f236a413f9a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.546607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e3c4-9a80-4f61-ac4a-9f236a413f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.567647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-8be1e3c4-9a80-4f61-ac4a-9f236a413f9a None None] MFA Rules not processed for user `41d9317e24714de1a310198bb35eb00b`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:25:49.568848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8be1e3c4-9a80-4f61-ac4a-9f236a413f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:49.578104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1187/2371] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:25:49 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:25:49.588565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe6a29-2ed9-48e9-a75e-eba9c539ea83 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:49.599519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe6a29-2ed9-48e9-a75e-eba9c539ea83 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:49.603052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6ecba-c4bf-4bc1-a9c1-f528951450b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:49.603648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6ecba-c4bf-4bc1-a9c1-f528951450b4 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:49.603924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6ecba-c4bf-4bc1-a9c1-f528951450b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:49.604191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6ecba-c4bf-4bc1-a9c1-f528951450b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:49.614861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6ecba-c4bf-4bc1-a9c1-f528951450b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:49.616284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6ecba-c4bf-4bc1-a9c1-f528951450b4 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:49.627398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1185/2372] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:49 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:50.085818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1e3c4-9a80-4f61-ac4a-9f236a413f9a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:50.096115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1e3c4-9a80-4f61-ac4a-9f236a413f9a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:50.099893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa2845-cbe8-442f-99f7-cad4b6220df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:50.100874 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2845-cbe8-442f-99f7-cad4b6220df0 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:50.101162 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2845-cbe8-442f-99f7-cad4b6220df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:50.101524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2845-cbe8-442f-99f7-cad4b6220df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:50.103380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-77fa2845-cbe8-442f-99f7-cad4b6220df0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:50.115610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa2845-cbe8-442f-99f7-cad4b6220df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:50.117057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa2845-cbe8-442f-99f7-cad4b6220df0 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:50.125709 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1188/2373] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:50 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:55.209106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6ecba-c4bf-4bc1-a9c1-f528951450b4 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:55.219633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6ecba-c4bf-4bc1-a9c1-f528951450b4 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:55.225384 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:55.226132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:55.226479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:55.226817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:55.228645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:55.255198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:55.256813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:55.268661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1186/2374] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:25:55 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:25:56.605330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0445-c87f-454c-b370-70e660e17aa0 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:56.605907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0445-c87f-454c-b370-70e660e17aa0 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:56.605907 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0445-c87f-454c-b370-70e660e17aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:56.628297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e5ff0445-c87f-454c-b370-70e660e17aa0 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:25:56.629438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e5ff0445-c87f-454c-b370-70e660e17aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:25:56.637018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1189/2375] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:25:56 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:25:56.649370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:25:56.660969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f772d4-67e3-4689-ab77-8044f2ebe77d service nova] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:25:56.666020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd68372-d093-4d38-a9f1-04ef7397545f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:25:56.666634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd68372-d093-4d38-a9f1-04ef7397545f service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:25:56.666917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd68372-d093-4d38-a9f1-04ef7397545f service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:25:56.667198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd68372-d093-4d38-a9f1-04ef7397545f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:25:56.675499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd68372-d093-4d38-a9f1-04ef7397545f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:25:56.677091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd68372-d093-4d38-a9f1-04ef7397545f service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:25:56.685154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1187/2376] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:25:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:26:06.621555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=27 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:26:06.682063 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=47 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:26:24.954041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4554-c7ea-4789-a9b4-6c95cbca9a6a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:26:24.954844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4554-c7ea-4789-a9b4-6c95cbca9a6a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:26:24.955128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4554-c7ea-4789-a9b4-6c95cbca9a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:26:24.978680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-db7b4554-c7ea-4789-a9b4-6c95cbca9a6a None None] MFA Rules not processed for user `91dd37f744a5476488afe91d97a406a1`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:26:24.979896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-db7b4554-c7ea-4789-a9b4-6c95cbca9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:26:25.037967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1190/2377] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:26:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:26:25.093995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd68372-d093-4d38-a9f1-04ef7397545f service swift] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:26:25.108981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd68372-d093-4d38-a9f1-04ef7397545f service swift] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:26:25.118817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:26:25.119719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:26:25.120237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:26:25.120736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:26:25.122803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:26:25.140997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:26:25.143220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:26:25.156388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1188/2378] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:26:25 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:26:27.781519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1344ffa-f5bd-487f-b038-7decfb3d0a78 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:26:27.782018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1344ffa-f5bd-487f-b038-7decfb3d0a78 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:26:27.782119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1344ffa-f5bd-487f-b038-7decfb3d0a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:26:27.821260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a1344ffa-f5bd-487f-b038-7decfb3d0a78 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:26:27.822572 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a1344ffa-f5bd-487f-b038-7decfb3d0a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:26:27.829843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1191/2379] 10.222.0.109 () {58 vars in 972 bytes} [Thu Aug 4 22:26:27 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:26:27.852973 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:26:27.864292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23c8f4-c319-4d2c-8ea0-747421f6c97f service nova] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:26:27.870161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e196a53-9170-4152-b034-4b22421b2f9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:26:27.870796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e196a53-9170-4152-b034-4b22421b2f9d service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:26:27.871070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e196a53-9170-4152-b034-4b22421b2f9d service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:26:27.871340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e196a53-9170-4152-b034-4b22421b2f9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:26:27.879235 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e196a53-9170-4152-b034-4b22421b2f9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:26:27.880837 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e196a53-9170-4152-b034-4b22421b2f9d service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:26:27.888725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1189/2380] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 22:26:32.797690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1344ffa-f5bd-487f-b038-7decfb3d0a78 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:26:32.810991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1344ffa-f5bd-487f-b038-7decfb3d0a78 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:26:32.816570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:26:32.817189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:26:32.817506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:26:32.817784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:26:32.819986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:26:32.878430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:26:32.880048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:26:32.890832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1192/2381] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:26:32 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:26:37.885603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=12 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:26:42.886290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=56 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:27:05.101382 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4f841-5e17-4911-9af9-16b13f53bf49 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:05.101926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4f841-5e17-4911-9af9-16b13f53bf49 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:05.102033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4f841-5e17-4911-9af9-16b13f53bf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:05.122210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ece4f841-5e17-4911-9af9-16b13f53bf49 None None] MFA Rules not processed for user `f3dab07011da408c97ed90d58f2c155b`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:27:05.123396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ece4f841-5e17-4911-9af9-16b13f53bf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:05.130237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1190/2382] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:27:05 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:27:12.595940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:12.610687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c57f4-fbd8-45e1-b638-09a469f36d06 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:12.618959 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade9d1-94d5-4771-b12e-30655be5ba2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:12.619677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade9d1-94d5-4771-b12e-30655be5ba2e service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:12.620081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade9d1-94d5-4771-b12e-30655be5ba2e service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:12.620419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade9d1-94d5-4771-b12e-30655be5ba2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:12.623033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-feade9d1-94d5-4771-b12e-30655be5ba2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:12.635880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade9d1-94d5-4771-b12e-30655be5ba2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:12.637607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade9d1-94d5-4771-b12e-30655be5ba2e service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:12.648653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1193/2383] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:27:12 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:27:15.116829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:27:22.643955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=6 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:27:38.921767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3929db-ec62-4a42-b4ee-f88a60b36eca None None] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:38.922296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3929db-ec62-4a42-b4ee-f88a60b36eca None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:38.922476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3929db-ec62-4a42-b4ee-f88a60b36eca None None] PATH_INFO: `/` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:38.924093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1191/2384] 10.222.0.109 () {54 vars in 926 bytes} [Thu Aug 4 22:27:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 22:27:38.950053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f97b7-9e3f-4069-a9fa-8762dc000349 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:38.950696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f97b7-9e3f-4069-a9fa-8762dc000349 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:38.950696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f97b7-9e3f-4069-a9fa-8762dc000349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:38.971270 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2b1f97b7-9e3f-4069-a9fa-8762dc000349 None None] MFA Rules not processed for user `0583af73d78a40deb2ca3af755928cee`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:27:38.972625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2b1f97b7-9e3f-4069-a9fa-8762dc000349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:38.984682 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1194/2385] 10.222.0.109 () {58 vars in 1023 bytes} [Thu Aug 4 22:27:38 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:27:38.988954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3929db-ec62-4a42-b4ee-f88a60b36eca None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:38.990717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fa3929db-ec62-4a42-b4ee-f88a60b36eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:39.005079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f97b7-9e3f-4069-a9fa-8762dc000349 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:39.006606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2b1f97b7-9e3f-4069-a9fa-8762dc000349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:39.024537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f97b7-9e3f-4069-a9fa-8762dc000349 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.032502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4d16f-9d76-400a-ac34-261f3ae2ed8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.033269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4d16f-9d76-400a-ac34-261f3ae2ed8c service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.033603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4d16f-9d76-400a-ac34-261f3ae2ed8c service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.033878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4d16f-9d76-400a-ac34-261f3ae2ed8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.044851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4d16f-9d76-400a-ac34-261f3ae2ed8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.047624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4d16f-9d76-400a-ac34-261f3ae2ed8c service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.058033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1195/2386] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:27:39 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:27:39.065300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3929db-ec62-4a42-b4ee-f88a60b36eca None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.071151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.071801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.072081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.072357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] PATH_INFO: `/v3/users/91dd37f744a5476488afe91d97a406a1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.073637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=91dd37f744a5476488afe91d97a406a1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.075871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.127965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91dd37f744a5476488afe91d97a406a1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.134702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91dd37f744a5476488afe91d97a406a1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.135434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91dd37f744a5476488afe91d97a406a1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.147588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91dd37f744a5476488afe91d97a406a1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.148239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-8617c76e-fa5e-4657-bca7-777bf90c93ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91dd37f744a5476488afe91d97a406a1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.161259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1192/2387] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:27:38 2022] DELETE /identity/v3/users/91dd37f744a5476488afe91d97a406a1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:27:39.337620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4d16f-9d76-400a-ac34-261f3ae2ed8c service placement] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:39.354561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4d16f-9d76-400a-ac34-261f3ae2ed8c service placement] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.411787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.412428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.412704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.412972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] PATH_INFO: `/v3/projects/260c01e60f5e48ac9420c388a958e0ee` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.414138 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=260c01e60f5e48ac9420c388a958e0ee)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.416037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.461989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260c01e60f5e48ac9420c388a958e0ee', 'request_id': 'req-83e245fe-1bbc-4000-a210-238e2bbc9d66'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.468208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260c01e60f5e48ac9420c388a958e0ee', 'request_id': 'req-83e245fe-1bbc-4000-a210-238e2bbc9d66'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.475619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260c01e60f5e48ac9420c388a958e0ee', 'request_id': 'req-83e245fe-1bbc-4000-a210-238e2bbc9d66'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.488239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] The token cache is being invalidate because project 260c01e60f5e48ac9420c388a958e0ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:27:39.488527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.489167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-83e245fe-1bbc-4000-a210-238e2bbc9d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:27:39.490730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1196/2388] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:27:39 2022] DELETE /identity/v3/projects/260c01e60f5e48ac9420c388a958e0ee => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:27:39.496282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda9962-71ce-474c-9138-5f85030f1f7f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.496641 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda9962-71ce-474c-9138-5f85030f1f7f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.496916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda9962-71ce-474c-9138-5f85030f1f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.516985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2dda9962-71ce-474c-9138-5f85030f1f7f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:27:39.518170 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2dda9962-71ce-474c-9138-5f85030f1f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:39.577087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1193/2389] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:27:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:27:39.643117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2041-ec5d-41d8-80f3-03cbf381fa41 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.643421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2041-ec5d-41d8-80f3-03cbf381fa41 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.644254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2041-ec5d-41d8-80f3-03cbf381fa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.670401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-287d2041-ec5d-41d8-80f3-03cbf381fa41 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:27:39.671620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-287d2041-ec5d-41d8-80f3-03cbf381fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:39.679866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1197/2390] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:27:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:27:39.684714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda9962-71ce-474c-9138-5f85030f1f7f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:39.695513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda9962-71ce-474c-9138-5f85030f1f7f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.700904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.701553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.701838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.702112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.702804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:27:39.703108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.704712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.717014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1194/2391] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:27:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:27:39.721039 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d2041-ec5d-41d8-80f3-03cbf381fa41 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:39.730431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d2041-ec5d-41d8-80f3-03cbf381fa41 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.735819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3b759-73e8-4c1b-84d3-4cca79c9c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.736424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3b759-73e8-4c1b-84d3-4cca79c9c692 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.736701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3b759-73e8-4c1b-84d3-4cca79c9c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.736969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3b759-73e8-4c1b-84d3-4cca79c9c692 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.737667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3b759-73e8-4c1b-84d3-4cca79c9c692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.739587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3b759-73e8-4c1b-84d3-4cca79c9c692 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.767250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1198/2392] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:27:39 2022] POST /identity/v3/projects => generated 374 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:27:39.771271 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:39.781936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0bed-f456-4123-aefd-a01c6dd218a9 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.785496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:27:39.838893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371d6d-ee28-434b-a45a-05d273eaf691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.839500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371d6d-ee28-434b-a45a-05d273eaf691 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.839774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371d6d-ee28-434b-a45a-05d273eaf691 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.840040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371d6d-ee28-434b-a45a-05d273eaf691 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.840701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3371d6d-ee28-434b-a45a-05d273eaf691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.842629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3371d6d-ee28-434b-a45a-05d273eaf691 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.863899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1195/2393] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:27:39 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:27:39.867912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3b759-73e8-4c1b-84d3-4cca79c9c692 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:39.878909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3b759-73e8-4c1b-84d3-4cca79c9c692 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.883625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13bfd-88e8-4767-a26a-9fac5376f1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.884266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13bfd-88e8-4767-a26a-9fac5376f1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.884575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13bfd-88e8-4767-a26a-9fac5376f1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.884847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13bfd-88e8-4767-a26a-9fac5376f1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.885573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a13bfd-88e8-4767-a26a-9fac5376f1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.887181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a13bfd-88e8-4767-a26a-9fac5376f1a0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.897776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1199/2394] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:27:39 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:27:39.901535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371d6d-ee28-434b-a45a-05d273eaf691 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:39.912175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371d6d-ee28-434b-a45a-05d273eaf691 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.916791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64bf89-6edd-4a1c-a54f-c0dbc533ee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.917396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64bf89-6edd-4a1c-a54f-c0dbc533ee07 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.917712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64bf89-6edd-4a1c-a54f-c0dbc533ee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.917989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64bf89-6edd-4a1c-a54f-c0dbc533ee07 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.919307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64bf89-6edd-4a1c-a54f-c0dbc533ee07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.920902 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64bf89-6edd-4a1c-a54f-c0dbc533ee07 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.931826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1196/2395] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:27:39 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:27:39.935982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13bfd-88e8-4767-a26a-9fac5376f1a0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:39.945924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13bfd-88e8-4767-a26a-9fac5376f1a0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:39.950567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:39.951248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.951525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.951796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] PATH_INFO: `/v3/projects/5bff3367439f45a7b7a420b66dd8574a/users/aaa106f1923b43dcb4ca4973815108c0/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:39.967172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bff3367439f45a7b7a420b66dd8574a, user_id=aaa106f1923b43dcb4ca4973815108c0, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:39.969943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:39.984012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1200/2396] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:27:39 2022] PUT /identity/v3/projects/5bff3367439f45a7b7a420b66dd8574a/users/aaa106f1923b43dcb4ca4973815108c0/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:27:39.989121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6cf6d-51fd-4b22-8aa5-1c46ee923562 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:39.989429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6cf6d-51fd-4b22-8aa5-1c46ee923562 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:39.989707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6cf6d-51fd-4b22-8aa5-1c46ee923562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:40.009077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-98d6cf6d-51fd-4b22-8aa5-1c46ee923562 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:27:40.010311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-98d6cf6d-51fd-4b22-8aa5-1c46ee923562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:40.069359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1197/2397] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:27:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:27:40.078969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:40.080656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:40.091160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:27:40.140687 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651fd1-46e8-4714-a6b9-025ec019d824 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:40.144364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f84a1-e54c-4017-aef7-679d9bf3bcf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:40.144977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f84a1-e54c-4017-aef7-679d9bf3bcf5 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:40.145256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f84a1-e54c-4017-aef7-679d9bf3bcf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:40.145561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f84a1-e54c-4017-aef7-679d9bf3bcf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:40.155064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f84a1-e54c-4017-aef7-679d9bf3bcf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:40.156492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f84a1-e54c-4017-aef7-679d9bf3bcf5 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:40.165775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1201/2398] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:27:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:27:44.147092 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1e2a-21ec-4dda-8c60-613241ae2b10 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:44.147626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1e2a-21ec-4dda-8c60-613241ae2b10 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:44.147626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1e2a-21ec-4dda-8c60-613241ae2b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:44.178016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-60fb1e2a-21ec-4dda-8c60-613241ae2b10 None None] MFA Rules not processed for user `aaa106f1923b43dcb4ca4973815108c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:27:44.179237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-60fb1e2a-21ec-4dda-8c60-613241ae2b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:44.255250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1198/2399] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:27:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:27:44.259773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f84a1-e54c-4017-aef7-679d9bf3bcf5 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:44.270407 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f84a1-e54c-4017-aef7-679d9bf3bcf5 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:44.275019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad19b2-9649-4f38-b3a9-d94a218d6a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:44.275635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad19b2-9649-4f38-b3a9-d94a218d6a4d admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:44.275909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad19b2-9649-4f38-b3a9-d94a218d6a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:44.276177 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad19b2-9649-4f38-b3a9-d94a218d6a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:44.276847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad19b2-9649-4f38-b3a9-d94a218d6a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:44.278985 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad19b2-9649-4f38-b3a9-d94a218d6a4d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:44.307311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1202/2400] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:27:44 2022] POST /identity/v3/projects => generated 372 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:27:44.312016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1e2a-21ec-4dda-8c60-613241ae2b10 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:44.323070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1e2a-21ec-4dda-8c60-613241ae2b10 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:44.336478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:27:44.385979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f5d0-b7a9-44ab-bc36-249688d8a431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:44.386651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f5d0-b7a9-44ab-bc36-249688d8a431 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:44.386926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f5d0-b7a9-44ab-bc36-249688d8a431 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:44.387201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f5d0-b7a9-44ab-bc36-249688d8a431 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:44.388056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598f5d0-b7a9-44ab-bc36-249688d8a431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:44.390051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598f5d0-b7a9-44ab-bc36-249688d8a431 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:44.416460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1199/2401] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:27:44 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:27:44.417577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad19b2-9649-4f38-b3a9-d94a218d6a4d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:44.428973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad19b2-9649-4f38-b3a9-d94a218d6a4d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:44.434485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a95ae-e9c6-4104-b458-beeed2ccd4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:44.435109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a95ae-e9c6-4104-b458-beeed2ccd4fb admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:44.435409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a95ae-e9c6-4104-b458-beeed2ccd4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:44.435683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a95ae-e9c6-4104-b458-beeed2ccd4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:44.436377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a95ae-e9c6-4104-b458-beeed2ccd4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:44.438062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a95ae-e9c6-4104-b458-beeed2ccd4fb admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:44.450036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1203/2402] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:27:44 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:27:44.454369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f5d0-b7a9-44ab-bc36-249688d8a431 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:44.466199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f5d0-b7a9-44ab-bc36-249688d8a431 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:44.470882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c2484-8932-40f9-a64d-b4e9bd94bfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:44.471588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c2484-8932-40f9-a64d-b4e9bd94bfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:44.471863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c2484-8932-40f9-a64d-b4e9bd94bfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:44.472134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c2484-8932-40f9-a64d-b4e9bd94bfbf admin admin] PATH_INFO: `/v3/projects/9ffc4f97d47d44d382250cb3a50e629e/users/eb32acc0ef7f474fa2e6ccbd93fb2c38/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:44.487527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c2484-8932-40f9-a64d-b4e9bd94bfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffc4f97d47d44d382250cb3a50e629e, user_id=eb32acc0ef7f474fa2e6ccbd93fb2c38, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:44.490250 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c2484-8932-40f9-a64d-b4e9bd94bfbf admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:44.503946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1200/2403] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:27:44 2022] PUT /identity/v3/projects/9ffc4f97d47d44d382250cb3a50e629e/users/eb32acc0ef7f474fa2e6ccbd93fb2c38/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:27:44.507965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a95ae-e9c6-4104-b458-beeed2ccd4fb admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:44.519062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a95ae-e9c6-4104-b458-beeed2ccd4fb admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:44.580764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a71ff8-48a4-45ef-8665-5017ccf0d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:44.581388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a71ff8-48a4-45ef-8665-5017ccf0d7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:44.581733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a71ff8-48a4-45ef-8665-5017ccf0d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:44.582002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a71ff8-48a4-45ef-8665-5017ccf0d7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:44.582689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a71ff8-48a4-45ef-8665-5017ccf0d7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:44.584279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a71ff8-48a4-45ef-8665-5017ccf0d7d0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:44.594549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1204/2404] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:27:44 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:27:44.598856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c2484-8932-40f9-a64d-b4e9bd94bfbf admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:44.609824 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c2484-8932-40f9-a64d-b4e9bd94bfbf admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:44.614606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:44.615273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:44.615577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:44.615850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] PATH_INFO: `/v3/projects/9ffc4f97d47d44d382250cb3a50e629e/users/eb32acc0ef7f474fa2e6ccbd93fb2c38/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:44.617881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffc4f97d47d44d382250cb3a50e629e, user_id=eb32acc0ef7f474fa2e6ccbd93fb2c38, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:44.620548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:44.631159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1201/2405] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:27:44 2022] PUT /identity/v3/projects/9ffc4f97d47d44d382250cb3a50e629e/users/eb32acc0ef7f474fa2e6ccbd93fb2c38/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:27:49.063210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290f34-2c01-44a0-9bf8-003f1e7213ac None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:49.064061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290f34-2c01-44a0-9bf8-003f1e7213ac None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:49.064531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290f34-2c01-44a0-9bf8-003f1e7213ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:49.122978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e4290f34-2c01-44a0-9bf8-003f1e7213ac None None] MFA Rules not processed for user `eb32acc0ef7f474fa2e6ccbd93fb2c38`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:27:49.124332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e4290f34-2c01-44a0-9bf8-003f1e7213ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:49.203456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:27:49.214211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1205/2406] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:27:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:27:49.233123 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:27:49.236043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:49.340960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5157f-5073-42a2-ba60-cd32d572cf67 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:27:49.347626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627247f-8a8d-45fb-b774-ab1757a0e4c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:27:49.348261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627247f-8a8d-45fb-b774-ab1757a0e4c5 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:49.348547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627247f-8a8d-45fb-b774-ab1757a0e4c5 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:49.348823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627247f-8a8d-45fb-b774-ab1757a0e4c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:49.411301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627247f-8a8d-45fb-b774-ab1757a0e4c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:27:49.413120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627247f-8a8d-45fb-b774-ab1757a0e4c5 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:27:49.432081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1202/2407] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:27:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:27:50.427037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbadcd-a89a-4477-815d-a1ffa6d3ab85 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:27:50.427540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbadcd-a89a-4477-815d-a1ffa6d3ab85 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:27:50.427660 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbadcd-a89a-4477-815d-a1ffa6d3ab85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:27:50.446854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-ffcbadcd-a89a-4477-815d-a1ffa6d3ab85 None None] MFA Rules not processed for user `aaa106f1923b43dcb4ca4973815108c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:27:50.448109 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ffcbadcd-a89a-4477-815d-a1ffa6d3ab85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:27:50.455695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1206/2408] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:27:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:27:59.420946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:00.441469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:09.491187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1b8cc-ed32-4161-83ed-cce4b927dcf5 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:09.491754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1b8cc-ed32-4161-83ed-cce4b927dcf5 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:09.491918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1b8cc-ed32-4161-83ed-cce4b927dcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:09.511359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-04a1b8cc-ed32-4161-83ed-cce4b927dcf5 None None] MFA Rules not processed for user `8c0aa2b9a5ac46f9bede8d007dcd3504`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:28:09.512595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-04a1b8cc-ed32-4161-83ed-cce4b927dcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:09.566840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1203/2409] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:28:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:28:09.903186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbadcd-a89a-4477-815d-a1ffa6d3ab85 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:09.962169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbadcd-a89a-4477-815d-a1ffa6d3ab85 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:09.966259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:09.967025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:09.967308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:09.968195 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:09.970221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:09.981622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:09.983889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:09.992782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1207/2410] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:28:09 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:28:19.559594 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=25 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:19.989357 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=26 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:25.699192 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6c0af-fb73-43a0-bfba-cb0c9e829ca2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:25.699721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6c0af-fb73-43a0-bfba-cb0c9e829ca2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:25.699924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6c0af-fb73-43a0-bfba-cb0c9e829ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:25.730415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-65e6c0af-fb73-43a0-bfba-cb0c9e829ca2 None None] MFA Rules not processed for user `8c50160e51b54cde9a3f617d71fbf1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:28:25.731625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-65e6c0af-fb73-43a0-bfba-cb0c9e829ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:25.812443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1204/2411] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:28:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:28:35.809885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=37 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:41.522424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:41.524192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:41.613028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75ea62-3d8e-40c7-8e9b-dcf1277fcd4c service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:41.616962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75ca96-df77-4396-9a06-9ad3b780c32f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:41.617573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ca96-df77-4396-9a06-9ad3b780c32f service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:41.617859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ca96-df77-4396-9a06-9ad3b780c32f service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:41.618136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ca96-df77-4396-9a06-9ad3b780c32f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:41.639428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75ca96-df77-4396-9a06-9ad3b780c32f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:41.641034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75ca96-df77-4396-9a06-9ad3b780c32f service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:41.653493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1208/2412] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:28:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:28:46.517620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6c0af-fb73-43a0-bfba-cb0c9e829ca2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:46.519032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-65e6c0af-fb73-43a0-bfba-cb0c9e829ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:46.615044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6c0af-fb73-43a0-bfba-cb0c9e829ca2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:46.620194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:46.621315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:46.621679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:46.621951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:46.623775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:46.635757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:46.638152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:46.649558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1205/2413] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:28:46 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:28:49.795822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75ca96-df77-4396-9a06-9ad3b780c32f service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:49.804669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75ca96-df77-4396-9a06-9ad3b780c32f service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:49.808314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:49.809232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:49.809597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:49.809872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:49.811606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:49.894273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:49.895623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:49.914520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1209/2414] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:28:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:28:53.015099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:53.028893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611457-0b91-4c60-9bf2-3855078819b8 service placement] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:53.033897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:53.034531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.035055 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.035339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.040802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:53.057682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:53.060363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:53.089800 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1206/2415] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:28:53 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:28:53.142754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:53.144336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:53.157210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093151c-2011-4c35-b7be-b67a549dbf66 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:53.163973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:53.164819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.165104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.165377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] PATH_INFO: `/v3/users/f3dab07011da408c97ed90d58f2c155b` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.166587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dab07011da408c97ed90d58f2c155b)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:53.169259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:53.216117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dab07011da408c97ed90d58f2c155b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.220538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:53.223868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dab07011da408c97ed90d58f2c155b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.232241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dab07011da408c97ed90d58f2c155b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.232958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dab07011da408c97ed90d58f2c155b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.245190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-33101671-af6f-4b9c-b6c3-ba1996fe8e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dab07011da408c97ed90d58f2c155b'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.253088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1210/2416] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:28:53 2022] DELETE /identity/v3/users/f3dab07011da408c97ed90d58f2c155b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:28:53.429689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:53.441513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840cc17-a094-438a-b447-de9dddb74f15 service placement] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:53.499560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:53.500284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.500561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.500829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] PATH_INFO: `/v3/projects/5116986e2edc4b5c92da0b77a497910a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.502000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5116986e2edc4b5c92da0b77a497910a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:53.503996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:53.551082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5116986e2edc4b5c92da0b77a497910a', 'request_id': 'req-2af0acd3-6fbc-477f-adf8-1883adda386f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.558691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5116986e2edc4b5c92da0b77a497910a', 'request_id': 'req-2af0acd3-6fbc-477f-adf8-1883adda386f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.560723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:53.565270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5116986e2edc4b5c92da0b77a497910a', 'request_id': 'req-2af0acd3-6fbc-477f-adf8-1883adda386f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.576955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] The token cache is being invalidate because project 5116986e2edc4b5c92da0b77a497910a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:28:53.577279 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.577954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2af0acd3-6fbc-477f-adf8-1883adda386f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:28:53.579567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1207/2417] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:28:53 2022] DELETE /identity/v3/projects/5116986e2edc4b5c92da0b77a497910a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:28:53.584542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceced1-6476-46dc-9183-8f291930e1bb None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.584829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceced1-6476-46dc-9183-8f291930e1bb None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.585103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceced1-6476-46dc-9183-8f291930e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.645526 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a7ceced1-6476-46dc-9183-8f291930e1bb None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:28:53.646776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a7ceced1-6476-46dc-9183-8f291930e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:53.705769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1211/2418] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:28:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:28:53.714662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35ee13-e877-455b-accb-b930f118fbc1 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.714950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35ee13-e877-455b-accb-b930f118fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.715229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35ee13-e877-455b-accb-b930f118fbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.735402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-bc35ee13-e877-455b-accb-b930f118fbc1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:28:53.736600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-bc35ee13-e877-455b-accb-b930f118fbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:53.744505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1208/2419] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:28:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:28:53.749299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ceced1-6476-46dc-9183-8f291930e1bb None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:53.759225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ceced1-6476-46dc-9183-8f291930e1bb None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:53.764305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:53.764905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.765181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.765486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.766173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:28:53.766470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:53.768043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:53.783274 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1212/2420] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:28:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:28:53.784476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35ee13-e877-455b-accb-b930f118fbc1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:53.794084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35ee13-e877-455b-accb-b930f118fbc1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:53.798977 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde09214-5f02-47ab-bda0-c086077e19f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:53.799588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09214-5f02-47ab-bda0-c086077e19f7 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.799911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09214-5f02-47ab-bda0-c086077e19f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.800188 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09214-5f02-47ab-bda0-c086077e19f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.800880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde09214-5f02-47ab-bda0-c086077e19f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:53.802820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde09214-5f02-47ab-bda0-c086077e19f7 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:53.829887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1209/2421] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:28:53 2022] POST /identity/v3/projects => generated 378 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:28:53.834037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:53.844532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db63be9-e919-407c-a4e9-202d25034161 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:53.900929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910278c-6992-4289-822e-4dcb60c6f749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:53.901672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910278c-6992-4289-822e-4dcb60c6f749 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.902043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910278c-6992-4289-822e-4dcb60c6f749 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.902312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910278c-6992-4289-822e-4dcb60c6f749 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.902991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910278c-6992-4289-822e-4dcb60c6f749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:53.904928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910278c-6992-4289-822e-4dcb60c6f749 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:53.926061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1213/2422] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:28:53 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:28:53.930190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde09214-5f02-47ab-bda0-c086077e19f7 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:53.941114 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde09214-5f02-47ab-bda0-c086077e19f7 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:53.946007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ff0cc-2797-4f29-8bd5-a537aec2828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:53.946622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ff0cc-2797-4f29-8bd5-a537aec2828b admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.946905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ff0cc-2797-4f29-8bd5-a537aec2828b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.947181 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ff0cc-2797-4f29-8bd5-a537aec2828b admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.947887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ff0cc-2797-4f29-8bd5-a537aec2828b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:53.950032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ff0cc-2797-4f29-8bd5-a537aec2828b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:53.959858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1210/2423] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:28:53 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:28:53.963722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910278c-6992-4289-822e-4dcb60c6f749 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:53.974485 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910278c-6992-4289-822e-4dcb60c6f749 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:53.979298 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c89e3-b80c-4e67-b071-10da0415900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:53.979962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c89e3-b80c-4e67-b071-10da0415900e admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:53.980236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c89e3-b80c-4e67-b071-10da0415900e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:53.980504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c89e3-b80c-4e67-b071-10da0415900e admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:53.981185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c89e3-b80c-4e67-b071-10da0415900e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:53.982812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c89e3-b80c-4e67-b071-10da0415900e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:53.993086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1214/2424] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:28:53 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:28:53.997485 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ff0cc-2797-4f29-8bd5-a537aec2828b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:54.007386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ff0cc-2797-4f29-8bd5-a537aec2828b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:54.012972 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:54.013617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:54.013900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:54.014177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] PATH_INFO: `/v3/projects/477c661a08dd46b2a20f53a7944361ef/users/890f3e80ae37401f8dad15d3d084f091/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:54.029637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=477c661a08dd46b2a20f53a7944361ef, user_id=890f3e80ae37401f8dad15d3d084f091, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:54.032337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:54.045246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1211/2425] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:28:53 2022] PUT /identity/v3/projects/477c661a08dd46b2a20f53a7944361ef/users/890f3e80ae37401f8dad15d3d084f091/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:28:54.050558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b92c2-435f-4dbf-aa37-8febdbea7de1 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:54.050848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b92c2-435f-4dbf-aa37-8febdbea7de1 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:54.051135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b92c2-435f-4dbf-aa37-8febdbea7de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:54.059203 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:54.070855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-212b92c2-435f-4dbf-aa37-8febdbea7de1 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:28:54.072695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-212b92c2-435f-4dbf-aa37-8febdbea7de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:54.139886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1215/2426] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:28:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:28:54.149847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:54.152289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:54.210217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbccc4f-f7e3-408b-bff3-b259c5c39e48 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:54.214851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:54.215496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:54.215783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:54.216055 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:54.225013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:54.226481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:54.236205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1212/2427] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:28:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:28:58.221473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6e8b-31ec-422f-8cfd-23b4afc9e14c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:58.221989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6e8b-31ec-422f-8cfd-23b4afc9e14c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:58.222121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6e8b-31ec-422f-8cfd-23b4afc9e14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:58.253028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-b8ef6e8b-31ec-422f-8cfd-23b4afc9e14c None None] MFA Rules not processed for user `890f3e80ae37401f8dad15d3d084f091`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:28:58.254197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b8ef6e8b-31ec-422f-8cfd-23b4afc9e14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:58.323635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1216/2428] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:28:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:28:58.344487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:28:58.346021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:58.477631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:28:58.490741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad54e-7a80-456e-8bc8-29c0281c1105 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:28:58.497648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6385f6-feab-445d-84ee-cdd9dbf14035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:28:58.498297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6385f6-feab-445d-84ee-cdd9dbf14035 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:58.498575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6385f6-feab-445d-84ee-cdd9dbf14035 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:58.498847 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6385f6-feab-445d-84ee-cdd9dbf14035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:58.508821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6385f6-feab-445d-84ee-cdd9dbf14035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:28:58.511059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6385f6-feab-445d-84ee-cdd9dbf14035 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:28:58.520645 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1213/2429] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:28:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:28:59.894591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e738b-b102-475f-917f-f5553c598e7f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:28:59.895085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e738b-b102-475f-917f-f5553c598e7f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:28:59.895200 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e738b-b102-475f-917f-f5553c598e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:28:59.914375 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-206e738b-b102-475f-917f-f5553c598e7f None None] MFA Rules not processed for user `890f3e80ae37401f8dad15d3d084f091`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:28:59.915567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-206e738b-b102-475f-917f-f5553c598e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:28:59.922387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1217/2430] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:28:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:29:08.516322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=7 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:29:09.909106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=60 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:30:05.408855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc67d3-ea3a-4c98-88af-f621f3964f32 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:05.409401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc67d3-ea3a-4c98-88af-f621f3964f32 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:05.409401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc67d3-ea3a-4c98-88af-f621f3964f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:05.443899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a8cc67d3-ea3a-4c98-88af-f621f3964f32 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:30:05.445803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a8cc67d3-ea3a-4c98-88af-f621f3964f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:05.506166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1214/2431] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:30:05 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:30:05.556388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e738b-b102-475f-917f-f5553c598e7f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:05.557851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-206e738b-b102-475f-917f-f5553c598e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:05.638134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e738b-b102-475f-917f-f5553c598e7f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:05.643211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:05.643824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:05.644106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:05.644376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:05.652453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:05.654084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:05.661983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1218/2432] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 22:30:11.406108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc67d3-ea3a-4c98-88af-f621f3964f32 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:11.418453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc67d3-ea3a-4c98-88af-f621f3964f32 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:11.422088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:11.422796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:11.423132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:11.423465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:11.425292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:11.441399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:11.443093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:11.456955 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1215/2433] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:30:11 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:30:15.078344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:15.079820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:15.092462 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246f96-80b8-41fd-882c-acedafd5daa6 service swift] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:15.096082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6433b62-6e3d-4fae-86f3-6235950dd734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:15.096690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6433b62-6e3d-4fae-86f3-6235950dd734 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:15.096969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6433b62-6e3d-4fae-86f3-6235950dd734 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:15.097239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6433b62-6e3d-4fae-86f3-6235950dd734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:15.099210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b6433b62-6e3d-4fae-86f3-6235950dd734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:15.114936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6433b62-6e3d-4fae-86f3-6235950dd734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:15.116376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6433b62-6e3d-4fae-86f3-6235950dd734 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:15.127559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1219/2434] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:30:15 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:30:17.435056 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:17.436528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:17.450864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c222197-328f-4b27-9b08-4d2c606ad191 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:17.456588 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:17.457232 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:17.457551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:17.457827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:17.459627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:17.539547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:17.541075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:17.561483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1216/2435] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:30:17 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:30:24.149471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296b4aa-e762-4ef6-803e-edabbc642de6 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:24.150000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296b4aa-e762-4ef6-803e-edabbc642de6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:24.150101 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296b4aa-e762-4ef6-803e-edabbc642de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:24.173374 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3296b4aa-e762-4ef6-803e-edabbc642de6 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:30:24.174881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3296b4aa-e762-4ef6-803e-edabbc642de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:24.189647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1220/2436] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:30:24 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:30:24.201333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:24.213574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728f52f-fd02-48e3-b894-66782c6a63b2 service nova] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:24.218515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:24.219163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:24.219439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:24.219709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:24.229967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:24.231564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:24.240984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1217/2437] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:30:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:30:26.766960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b365d-439d-4b76-83d4-8592f2ad5de3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:26.767823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b365d-439d-4b76-83d4-8592f2ad5de3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:26.768284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b365d-439d-4b76-83d4-8592f2ad5de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:26.819635 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-163b365d-439d-4b76-83d4-8592f2ad5de3 None None] MFA Rules not processed for user `8c50160e51b54cde9a3f617d71fbf1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:30:26.820859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-163b365d-439d-4b76-83d4-8592f2ad5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:26.879537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1221/2438] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:30:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:30:27.629267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:27.630967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:27.693258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:30:27.736124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081dc74-884a-4305-972f-1e08e97ddd98 service swift] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:27.742266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314126f9-12d4-4f50-b7cf-4fac87706d73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:27.742888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314126f9-12d4-4f50-b7cf-4fac87706d73 service placement] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:27.743298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314126f9-12d4-4f50-b7cf-4fac87706d73 service placement] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:27.743573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314126f9-12d4-4f50-b7cf-4fac87706d73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:27.745383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-314126f9-12d4-4f50-b7cf-4fac87706d73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:27.760477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314126f9-12d4-4f50-b7cf-4fac87706d73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:27.762118 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314126f9-12d4-4f50-b7cf-4fac87706d73 service placement] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:27.773686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1218/2439] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:30:27 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:30:30.930789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b365d-439d-4b76-83d4-8592f2ad5de3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:30.937983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=76 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:30:30.951456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b365d-439d-4b76-83d4-8592f2ad5de3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:30.957637 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:30.958364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:30.958802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:30.959147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:30.961067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:30.974810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:30.976550 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:31.001573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1222/2440] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:30:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:30:37.769074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=14 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:30:40.998500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=9 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:30:46.225584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509eb95-f287-4b94-9937-c336d67800d2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:46.226071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509eb95-f287-4b94-9937-c336d67800d2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:46.226185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509eb95-f287-4b94-9937-c336d67800d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:46.275416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8509eb95-f287-4b94-9937-c336d67800d2 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:30:46.276636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8509eb95-f287-4b94-9937-c336d67800d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:46.293526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1219/2441] 10.222.0.109 () {58 vars in 972 bytes} [Thu Aug 4 22:30:46 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:30:46.306269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:46.327644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485ce23-c7c2-447d-ae0e-83b7451e390b service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:46.332725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a02f7-bca0-4556-b1f6-dd8a286a53ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:46.333333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a02f7-bca0-4556-b1f6-dd8a286a53ed service swift] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:46.333647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a02f7-bca0-4556-b1f6-dd8a286a53ed service swift] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:46.333921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a02f7-bca0-4556-b1f6-dd8a286a53ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:46.342021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a02f7-bca0-4556-b1f6-dd8a286a53ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:46.343596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a02f7-bca0-4556-b1f6-dd8a286a53ed service swift] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:46.352140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1223/2442] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:30:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:30:50.718476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509eb95-f287-4b94-9937-c336d67800d2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:30:50.728138 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509eb95-f287-4b94-9937-c336d67800d2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:30:50.731774 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:30:50.732332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:30:50.732612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:30:50.732885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:30:50.734717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:30:50.745198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:30:50.746687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:30:50.755836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1220/2443] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:30:50 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:30:56.348938 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=10 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:31:00.751230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=24 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:31:24.904304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a02f7-bca0-4556-b1f6-dd8a286a53ed service swift] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:31:24.914727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a02f7-bca0-4556-b1f6-dd8a286a53ed service swift] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:31:24.918299 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:31:24.918932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:31:24.919418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:31:24.919694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:31:24.921453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:31:24.931442 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:31:24.932872 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:31:24.941642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1224/2444] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:31:24 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:31:25.044834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:31:25.054253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722466a-1086-4233-9d74-dafec7fc87dd service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:31:25.057939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:31:25.058556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:31:25.058835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:31:25.059110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:31:25.060919 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:31:25.078672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:31:25.080099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:31:25.090854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1221/2445] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:31:25 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:31:27.212739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:31:27.228679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f2ff3-1564-4712-a5f4-1c7d97bd2f75 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:31:27.235139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:31:27.235997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:31:27.236436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:31:27.236901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:31:27.239586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:31:27.255599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:31:27.257509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:31:27.269305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1225/2446] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:31:27 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:31:35.086241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=6 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:31:37.264762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=12 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:31:55.307932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:31:55.322849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6efa18-63f7-4ce7-8851-6db4345d17aa service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:31:55.328611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:31:55.329438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:31:55.329946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:31:55.330224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:31:55.332922 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:31:55.350699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:31:55.352443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:31:55.365336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1222/2447] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:31:55 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:31:59.009199 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:31:59.019965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94f03b-06a2-4309-acba-e2d4e99829b1 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:31:59.023662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:31:59.024291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:31:59.024567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:31:59.024864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:31:59.026714 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:31:59.040028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:31:59.041536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:31:59.054016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1226/2448] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:31:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:02.540316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:02.541897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:02.556596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b93a72-f67f-40fc-a416-206cae5ccdc7 service nova] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:02.561223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:02.561893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:02.562171 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:02.562561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] PATH_INFO: `/v3/users/8c0aa2b9a5ac46f9bede8d007dcd3504` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:02.564367 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0aa2b9a5ac46f9bede8d007dcd3504)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:02.566323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:02.611841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0aa2b9a5ac46f9bede8d007dcd3504'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.618768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0aa2b9a5ac46f9bede8d007dcd3504'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.619374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0aa2b9a5ac46f9bede8d007dcd3504'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.631740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0aa2b9a5ac46f9bede8d007dcd3504'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.632375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-66e51799-4457-4fa0-8eb5-f38bc7b3da31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0aa2b9a5ac46f9bede8d007dcd3504'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.644166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1223/2449] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:02 2022] DELETE /identity/v3/users/8c0aa2b9a5ac46f9bede8d007dcd3504 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:02.816114 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:02.826131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b4fd4-e84d-46e4-be33-e175dd152fed service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:02.885123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:02.885797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:02.886082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:02.886358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] PATH_INFO: `/v3/projects/21d013d03bbc4f8ead46aca55d441eb1` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:02.887520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d013d03bbc4f8ead46aca55d441eb1)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:02.889450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:02.937170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d013d03bbc4f8ead46aca55d441eb1', 'request_id': 'req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.943465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d013d03bbc4f8ead46aca55d441eb1', 'request_id': 'req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.950625 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d013d03bbc4f8ead46aca55d441eb1', 'request_id': 'req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.962404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] The token cache is being invalidate because project 21d013d03bbc4f8ead46aca55d441eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:02.962690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.963321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b3dccdba-9fca-48e7-8ed8-bdaeb99c6b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:02.964856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1227/2450] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:02 2022] DELETE /identity/v3/projects/21d013d03bbc4f8ead46aca55d441eb1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:02.971150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b2642-721e-48c1-873c-52b5dae22957 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:02.971438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b2642-721e-48c1-873c-52b5dae22957 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:02.971715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b2642-721e-48c1-873c-52b5dae22957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:02.991813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9f8b2642-721e-48c1-873c-52b5dae22957 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:02.993013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9f8b2642-721e-48c1-873c-52b5dae22957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:03.057351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1224/2451] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:03.059525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6db5df-759a-4323-a025-ede70fa28b42 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.059814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6db5df-759a-4323-a025-ede70fa28b42 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.060112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6db5df-759a-4323-a025-ede70fa28b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.082697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2e6db5df-759a-4323-a025-ede70fa28b42 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:03.083914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2e6db5df-759a-4323-a025-ede70fa28b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:03.091997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1228/2452] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:03.097303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b2642-721e-48c1-873c-52b5dae22957 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:03.108237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b2642-721e-48c1-873c-52b5dae22957 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:03.113769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:03.114064 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.114343 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.114612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.115305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:03.115606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:03.117238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:03.130070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1225/2453] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:32:03.134407 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6db5df-759a-4323-a025-ede70fa28b42 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:03.144678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6db5df-759a-4323-a025-ede70fa28b42 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:03.149331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b37bf-54fe-4b09-8e7a-5924701ef650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:03.149976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b37bf-54fe-4b09-8e7a-5924701ef650 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.150255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b37bf-54fe-4b09-8e7a-5924701ef650 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.150523 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b37bf-54fe-4b09-8e7a-5924701ef650 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.151182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b37bf-54fe-4b09-8e7a-5924701ef650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:03.153074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b37bf-54fe-4b09-8e7a-5924701ef650 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:03.195389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1229/2454] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:03 2022] POST /identity/v3/projects => generated 404 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:03.196739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:03.207994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce96d80-3cbb-4ae2-a04d-ba1baefc7105 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:03.253701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:03.268182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0faba-97e7-44af-8fd4-8cbabed8af57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:03.268795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0faba-97e7-44af-8fd4-8cbabed8af57 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.269072 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0faba-97e7-44af-8fd4-8cbabed8af57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.269341 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0faba-97e7-44af-8fd4-8cbabed8af57 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.270031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0faba-97e7-44af-8fd4-8cbabed8af57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:03.271960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0faba-97e7-44af-8fd4-8cbabed8af57 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:03.293374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1226/2455] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:03 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:03.297521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b37bf-54fe-4b09-8e7a-5924701ef650 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:03.307673 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b37bf-54fe-4b09-8e7a-5924701ef650 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:03.312239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fa4f4-54ba-4bc9-971d-d6e723fc4709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:03.312818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa4f4-54ba-4bc9-971d-d6e723fc4709 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.313091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa4f4-54ba-4bc9-971d-d6e723fc4709 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.313472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa4f4-54ba-4bc9-971d-d6e723fc4709 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.314152 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fa4f4-54ba-4bc9-971d-d6e723fc4709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:03.315736 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fa4f4-54ba-4bc9-971d-d6e723fc4709 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:03.325527 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1230/2456] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:03 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:03.329952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0faba-97e7-44af-8fd4-8cbabed8af57 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:03.340144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0faba-97e7-44af-8fd4-8cbabed8af57 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:03.345936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af984a9b-930c-476c-ac36-0d781a09da5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:03.346547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984a9b-930c-476c-ac36-0d781a09da5f admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.346829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984a9b-930c-476c-ac36-0d781a09da5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.347103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984a9b-930c-476c-ac36-0d781a09da5f admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.347797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af984a9b-930c-476c-ac36-0d781a09da5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:03.349396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af984a9b-930c-476c-ac36-0d781a09da5f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:03.359275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1227/2457] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:03 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:03.363260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fa4f4-54ba-4bc9-971d-d6e723fc4709 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:03.373123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fa4f4-54ba-4bc9-971d-d6e723fc4709 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:03.377975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:03.378630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.378911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.379189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] PATH_INFO: `/v3/projects/488a09c2f7d9441889f2cdef25adf94d/users/a555245f9076436fa3c2552ebbd262d1/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.396694 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488a09c2f7d9441889f2cdef25adf94d, user_id=a555245f9076436fa3c2552ebbd262d1, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:03.399628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:03.413386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1231/2458] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:03 2022] PUT /identity/v3/projects/488a09c2f7d9441889f2cdef25adf94d/users/a555245f9076436fa3c2552ebbd262d1/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:03.418449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c3287-cbf3-4c7f-80a8-308f8b800722 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.418732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c3287-cbf3-4c7f-80a8-308f8b800722 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.419009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c3287-cbf3-4c7f-80a8-308f8b800722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.438312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-757c3287-cbf3-4c7f-80a8-308f8b800722 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:03.439507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-757c3287-cbf3-4c7f-80a8-308f8b800722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:03.496684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1228/2459] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:03.507007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:03.508721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:03.515156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:03.578915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999a77a-136f-4d4e-8f2a-926094d9cf53 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:03.583703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f308e9-414e-4956-bdca-6dec9cd83c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:03.584324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f308e9-414e-4956-bdca-6dec9cd83c58 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:03.584607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f308e9-414e-4956-bdca-6dec9cd83c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:03.584881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f308e9-414e-4956-bdca-6dec9cd83c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:03.594802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f308e9-414e-4956-bdca-6dec9cd83c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:03.596237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f308e9-414e-4956-bdca-6dec9cd83c58 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:03.608909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1232/2460] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:03 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:04.245047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594eca8-da94-4ed2-846c-142dae950801 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:04.245635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594eca8-da94-4ed2-846c-142dae950801 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:04.245635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594eca8-da94-4ed2-846c-142dae950801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:04.275904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-4594eca8-da94-4ed2-846c-142dae950801 None None] MFA Rules not processed for user `a555245f9076436fa3c2552ebbd262d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:04.277198 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4594eca8-da94-4ed2-846c-142dae950801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:04.344533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1229/2461] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:04 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:04.349113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f308e9-414e-4956-bdca-6dec9cd83c58 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:04.359057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f308e9-414e-4956-bdca-6dec9cd83c58 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:04.363645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3c8f-810e-440b-ac80-a1126812edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:04.364364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3c8f-810e-440b-ac80-a1126812edbf admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:04.364732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3c8f-810e-440b-ac80-a1126812edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:04.365038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3c8f-810e-440b-ac80-a1126812edbf admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:04.365823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc3c8f-810e-440b-ac80-a1126812edbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:04.367713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc3c8f-810e-440b-ac80-a1126812edbf admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:04.397789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1233/2462] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:04 2022] POST /identity/v3/projects => generated 406 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:04.398930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594eca8-da94-4ed2-846c-142dae950801 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:04.410758 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594eca8-da94-4ed2-846c-142dae950801 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:04.460110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:04.468020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcf051-89f1-4e6f-bdba-681e70330e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:04.469459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcf051-89f1-4e6f-bdba-681e70330e26 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:04.469733 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcf051-89f1-4e6f-bdba-681e70330e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:04.470101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcf051-89f1-4e6f-bdba-681e70330e26 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:04.470753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcf051-89f1-4e6f-bdba-681e70330e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:04.472670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcf051-89f1-4e6f-bdba-681e70330e26 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:04.493668 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1230/2463] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:04 2022] POST /identity/v3/users => generated 310 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:04.497608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3c8f-810e-440b-ac80-a1126812edbf admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:04.507799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3c8f-810e-440b-ac80-a1126812edbf admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:04.512495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27142c5-de24-4088-a4e3-9a1fb502b971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:04.513131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27142c5-de24-4088-a4e3-9a1fb502b971 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:04.513434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27142c5-de24-4088-a4e3-9a1fb502b971 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:04.513722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27142c5-de24-4088-a4e3-9a1fb502b971 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:04.514412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27142c5-de24-4088-a4e3-9a1fb502b971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:04.516548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27142c5-de24-4088-a4e3-9a1fb502b971 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:04.528570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1234/2464] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:04 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:04.533087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcf051-89f1-4e6f-bdba-681e70330e26 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:04.543399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcf051-89f1-4e6f-bdba-681e70330e26 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:04.549099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648d7cb-31a2-4e4a-8d06-62fc3446ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:04.549790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648d7cb-31a2-4e4a-8d06-62fc3446ed9f admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:04.550071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648d7cb-31a2-4e4a-8d06-62fc3446ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:04.550345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648d7cb-31a2-4e4a-8d06-62fc3446ed9f admin admin] PATH_INFO: `/v3/projects/257f7e0bc4dc46148dff53bd42dfe54e/users/8c704060d1b84281b9061015a566d652/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:04.565851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648d7cb-31a2-4e4a-8d06-62fc3446ed9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=257f7e0bc4dc46148dff53bd42dfe54e, user_id=8c704060d1b84281b9061015a566d652, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:04.568514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648d7cb-31a2-4e4a-8d06-62fc3446ed9f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:04.581419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1231/2465] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:04 2022] PUT /identity/v3/projects/257f7e0bc4dc46148dff53bd42dfe54e/users/8c704060d1b84281b9061015a566d652/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:04.585340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27142c5-de24-4088-a4e3-9a1fb502b971 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:04.595689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27142c5-de24-4088-a4e3-9a1fb502b971 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:04.659730 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbc696-2416-4d21-9381-c448099491af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:04.660268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc696-2416-4d21-9381-c448099491af admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:04.660545 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc696-2416-4d21-9381-c448099491af admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:04.660811 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc696-2416-4d21-9381-c448099491af admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:04.661523 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbc696-2416-4d21-9381-c448099491af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:04.663108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbc696-2416-4d21-9381-c448099491af admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:04.673616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1235/2466] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:04 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:04.677710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648d7cb-31a2-4e4a-8d06-62fc3446ed9f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:04.689373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648d7cb-31a2-4e4a-8d06-62fc3446ed9f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:04.694104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d4b73-f45c-4868-b08e-ab547c8b128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:04.694770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d4b73-f45c-4868-b08e-ab547c8b128b admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:04.695046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d4b73-f45c-4868-b08e-ab547c8b128b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:04.695317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d4b73-f45c-4868-b08e-ab547c8b128b admin admin] PATH_INFO: `/v3/projects/257f7e0bc4dc46148dff53bd42dfe54e/users/8c704060d1b84281b9061015a566d652/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:04.697263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d4b73-f45c-4868-b08e-ab547c8b128b admin admin] RBAC: Authorizing `identity:create_grant(project_id=257f7e0bc4dc46148dff53bd42dfe54e, user_id=8c704060d1b84281b9061015a566d652, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:04.700033 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d4b73-f45c-4868-b08e-ab547c8b128b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:04.710807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1232/2467] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:04 2022] PUT /identity/v3/projects/257f7e0bc4dc46148dff53bd42dfe54e/users/8c704060d1b84281b9061015a566d652/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:05.350542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a7dac-06f6-4193-9b02-b95f7749d56f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:05.351124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a7dac-06f6-4193-9b02-b95f7749d56f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:05.351124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a7dac-06f6-4193-9b02-b95f7749d56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:05.383288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-420a7dac-06f6-4193-9b02-b95f7749d56f None None] MFA Rules not processed for user `8c704060d1b84281b9061015a566d652`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:05.384509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-420a7dac-06f6-4193-9b02-b95f7749d56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:05.452630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:05.464945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1236/2468] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:05 2022] POST /identity/v3/auth/tokens => generated 3359 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:06.498520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d4b73-f45c-4868-b08e-ab547c8b128b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:06.562042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d4b73-f45c-4868-b08e-ab547c8b128b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:06.567424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:06.568066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:06.568349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:06.568625 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] PATH_INFO: `/v3/users/a555245f9076436fa3c2552ebbd262d1` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:06.569830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a555245f9076436fa3c2552ebbd262d1)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:06.571777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:06.616942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a555245f9076436fa3c2552ebbd262d1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.623562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a555245f9076436fa3c2552ebbd262d1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.624273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a555245f9076436fa3c2552ebbd262d1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.639652 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a555245f9076436fa3c2552ebbd262d1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.641148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a555245f9076436fa3c2552ebbd262d1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.653533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1233/2469] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:06 2022] DELETE /identity/v3/users/a555245f9076436fa3c2552ebbd262d1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:06.824575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a7dac-06f6-4193-9b02-b95f7749d56f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:06.836284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a7dac-06f6-4193-9b02-b95f7749d56f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:06.899158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:06.899727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:06.900008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:06.900283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] PATH_INFO: `/v3/projects/488a09c2f7d9441889f2cdef25adf94d` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:06.901464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=488a09c2f7d9441889f2cdef25adf94d)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:06.903362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:06.947974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '488a09c2f7d9441889f2cdef25adf94d', 'request_id': 'req-8dc041b9-a107-4dfd-9c21-ae3152e569d6'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.954856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '488a09c2f7d9441889f2cdef25adf94d', 'request_id': 'req-8dc041b9-a107-4dfd-9c21-ae3152e569d6'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.962042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '488a09c2f7d9441889f2cdef25adf94d', 'request_id': 'req-8dc041b9-a107-4dfd-9c21-ae3152e569d6'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.973611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] The token cache is being invalidate because project 488a09c2f7d9441889f2cdef25adf94d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:06.973893 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.974520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:06.976013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1237/2470] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:06 2022] DELETE /identity/v3/projects/488a09c2f7d9441889f2cdef25adf94d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:06.980153 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:06.981598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:06.994355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e24007-9c84-412d-b313-9820f6a06d2f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:07.017899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:07.052256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:07.052900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.053195 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.053454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] PATH_INFO: `/v3/users/8c704060d1b84281b9061015a566d652` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.054684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c704060d1b84281b9061015a566d652)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:07.056613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:07.102042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c704060d1b84281b9061015a566d652'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.109990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c704060d1b84281b9061015a566d652'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.110713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c704060d1b84281b9061015a566d652'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.122469 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c704060d1b84281b9061015a566d652'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.123097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-27b27ccd-262c-4df4-aa42-f35d8e5de8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c704060d1b84281b9061015a566d652'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.135599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1234/2471] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:06 2022] DELETE /identity/v3/users/8c704060d1b84281b9061015a566d652 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:07.310518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:07.311974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:07.325256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc041b9-a107-4dfd-9c21-ae3152e569d6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:07.384287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:07.384942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.385225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.385537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] PATH_INFO: `/v3/projects/257f7e0bc4dc46148dff53bd42dfe54e` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.386689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=257f7e0bc4dc46148dff53bd42dfe54e)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:07.388627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:07.420103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:07.436192 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257f7e0bc4dc46148dff53bd42dfe54e', 'request_id': 'req-34a56700-c714-4dac-a15f-0a2a5342b100'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.442288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257f7e0bc4dc46148dff53bd42dfe54e', 'request_id': 'req-34a56700-c714-4dac-a15f-0a2a5342b100'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.449716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257f7e0bc4dc46148dff53bd42dfe54e', 'request_id': 'req-34a56700-c714-4dac-a15f-0a2a5342b100'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.461937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] The token cache is being invalidate because project 257f7e0bc4dc46148dff53bd42dfe54e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:07.462222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.462861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-34a56700-c714-4dac-a15f-0a2a5342b100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:07.464376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1238/2472] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:07 2022] DELETE /identity/v3/projects/257f7e0bc4dc46148dff53bd42dfe54e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:07.470470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d806a30-8e55-4979-8d9c-0a1078d76920 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.470750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d806a30-8e55-4979-8d9c-0a1078d76920 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.471029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d806a30-8e55-4979-8d9c-0a1078d76920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.491074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0d806a30-8e55-4979-8d9c-0a1078d76920 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:07.492290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0d806a30-8e55-4979-8d9c-0a1078d76920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:07.551277 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1235/2473] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:07.559314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e27dc4-f744-45f7-968c-2812502e14b3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.559675 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e27dc4-f744-45f7-968c-2812502e14b3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.560052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e27dc4-f744-45f7-968c-2812502e14b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.580495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-51e27dc4-f744-45f7-968c-2812502e14b3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:07.581717 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-51e27dc4-f744-45f7-968c-2812502e14b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:07.589759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1239/2474] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:32:07.594784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d806a30-8e55-4979-8d9c-0a1078d76920 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:07.604616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d806a30-8e55-4979-8d9c-0a1078d76920 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:07.609660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:07.610331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.610634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.610904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.611591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:07.611887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:07.613556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:07.625373 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1236/2475] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:32:07.629720 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e27dc4-f744-45f7-968c-2812502e14b3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:07.638739 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e27dc4-f744-45f7-968c-2812502e14b3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:07.643699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044dc1d-70c3-4639-9dc9-f3901ce82aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:07.644302 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044dc1d-70c3-4639-9dc9-f3901ce82aaf admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.644577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044dc1d-70c3-4639-9dc9-f3901ce82aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.644848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044dc1d-70c3-4639-9dc9-f3901ce82aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.645538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044dc1d-70c3-4639-9dc9-f3901ce82aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:07.647413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044dc1d-70c3-4639-9dc9-f3901ce82aaf admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:07.673651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1240/2476] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:07 2022] POST /identity/v3/projects => generated 368 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:07.677883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:07.688759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4de62-68aa-4613-959c-05a45173afd1 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:07.717151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:07.745530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1e4ef-220c-49b2-bc32-8e7a48e7f7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:07.746143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e4ef-220c-49b2-bc32-8e7a48e7f7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.746417 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e4ef-220c-49b2-bc32-8e7a48e7f7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.746700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e4ef-220c-49b2-bc32-8e7a48e7f7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.747345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1e4ef-220c-49b2-bc32-8e7a48e7f7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:07.749244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1e4ef-220c-49b2-bc32-8e7a48e7f7d1 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:07.769925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1237/2477] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:07 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:07.773975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044dc1d-70c3-4639-9dc9-f3901ce82aaf admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:07.784350 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044dc1d-70c3-4639-9dc9-f3901ce82aaf admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:07.789100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba717d0-6ef5-4c3b-83bc-c1f17396824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:07.789743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba717d0-6ef5-4c3b-83bc-c1f17396824c admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.790042 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba717d0-6ef5-4c3b-83bc-c1f17396824c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.790309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba717d0-6ef5-4c3b-83bc-c1f17396824c admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.791023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba717d0-6ef5-4c3b-83bc-c1f17396824c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:07.792620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba717d0-6ef5-4c3b-83bc-c1f17396824c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:07.803052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1241/2478] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:07 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:07.806672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1e4ef-220c-49b2-bc32-8e7a48e7f7d1 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:07.816858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1e4ef-220c-49b2-bc32-8e7a48e7f7d1 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:07.821313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f141c-a8fb-4539-bd01-e4ad02ce07aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:07.821938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f141c-a8fb-4539-bd01-e4ad02ce07aa admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.822213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f141c-a8fb-4539-bd01-e4ad02ce07aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.822481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f141c-a8fb-4539-bd01-e4ad02ce07aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.823157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f141c-a8fb-4539-bd01-e4ad02ce07aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:07.824738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f141c-a8fb-4539-bd01-e4ad02ce07aa admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:07.835184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1238/2479] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:07 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:07.839075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba717d0-6ef5-4c3b-83bc-c1f17396824c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:07.850189 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba717d0-6ef5-4c3b-83bc-c1f17396824c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:07.854825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b46aea-b5a0-4048-8878-209aee6d316a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:07.855481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b46aea-b5a0-4048-8878-209aee6d316a admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.855764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b46aea-b5a0-4048-8878-209aee6d316a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.856037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b46aea-b5a0-4048-8878-209aee6d316a admin admin] PATH_INFO: `/v3/projects/6ce92c88991f4d7fbb66092136c2bbfd/users/7a4b0f60a44d4501a5d2e2279cc75311/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.871175 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b46aea-b5a0-4048-8878-209aee6d316a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce92c88991f4d7fbb66092136c2bbfd, user_id=7a4b0f60a44d4501a5d2e2279cc75311, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:07.873855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b46aea-b5a0-4048-8878-209aee6d316a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:07.887559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1242/2480] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:07 2022] PUT /identity/v3/projects/6ce92c88991f4d7fbb66092136c2bbfd/users/7a4b0f60a44d4501a5d2e2279cc75311/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:07.899823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2cab2-147c-4c07-a5ff-f0c3fd79ad13 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:07.900113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2cab2-147c-4c07-a5ff-f0c3fd79ad13 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:07.900391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2cab2-147c-4c07-a5ff-f0c3fd79ad13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:07.927815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-22c2cab2-147c-4c07-a5ff-f0c3fd79ad13 None None] MFA Rules not processed for user `7a4b0f60a44d4501a5d2e2279cc75311`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:07.929014 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-22c2cab2-147c-4c07-a5ff-f0c3fd79ad13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:07.991371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1239/2481] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:07.995723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b46aea-b5a0-4048-8878-209aee6d316a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:08.055746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b46aea-b5a0-4048-8878-209aee6d316a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:08.060265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a3e7-70bf-40c5-a0d3-66603fa56e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:08.060863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a3e7-70bf-40c5-a0d3-66603fa56e1c admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:08.061139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a3e7-70bf-40c5-a0d3-66603fa56e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:08.061424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a3e7-70bf-40c5-a0d3-66603fa56e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:08.062085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a3e7-70bf-40c5-a0d3-66603fa56e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:08.063997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a3e7-70bf-40c5-a0d3-66603fa56e1c admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:08.088708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1243/2482] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:07 2022] POST /identity/v3/projects => generated 370 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:08.093106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2cab2-147c-4c07-a5ff-f0c3fd79ad13 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:08.104833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2cab2-147c-4c07-a5ff-f0c3fd79ad13 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:08.162511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e65573-0cce-4299-b8e5-8a12e9f684ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:08.163135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65573-0cce-4299-b8e5-8a12e9f684ec admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:08.163415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65573-0cce-4299-b8e5-8a12e9f684ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:08.163686 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65573-0cce-4299-b8e5-8a12e9f684ec admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:08.164369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e65573-0cce-4299-b8e5-8a12e9f684ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:08.166332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e65573-0cce-4299-b8e5-8a12e9f684ec admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:08.187706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1240/2483] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:08 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:08.191836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a3e7-70bf-40c5-a0d3-66603fa56e1c admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:08.202172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a3e7-70bf-40c5-a0d3-66603fa56e1c admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:08.207410 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434f46-b6f6-4ac5-899e-772ed30c8a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:08.208015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434f46-b6f6-4ac5-899e-772ed30c8a37 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:08.208292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434f46-b6f6-4ac5-899e-772ed30c8a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:08.208561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434f46-b6f6-4ac5-899e-772ed30c8a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:08.209246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a434f46-b6f6-4ac5-899e-772ed30c8a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:08.210871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a434f46-b6f6-4ac5-899e-772ed30c8a37 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:08.222435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1244/2484] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:08 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:08.226496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e65573-0cce-4299-b8e5-8a12e9f684ec admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:08.237675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e65573-0cce-4299-b8e5-8a12e9f684ec admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:08.242513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196fb11-9375-427b-90b0-d060e421c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:08.242928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196fb11-9375-427b-90b0-d060e421c124 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:08.243206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196fb11-9375-427b-90b0-d060e421c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:08.243479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196fb11-9375-427b-90b0-d060e421c124 admin admin] PATH_INFO: `/v3/projects/95afef17853f4a2b9b8c659da2683803/users/d13aa2d1db134687b8f7714009e6c679/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:08.258592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196fb11-9375-427b-90b0-d060e421c124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95afef17853f4a2b9b8c659da2683803, user_id=d13aa2d1db134687b8f7714009e6c679, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:08.261256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196fb11-9375-427b-90b0-d060e421c124 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:08.274880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1241/2485] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:08 2022] PUT /identity/v3/projects/95afef17853f4a2b9b8c659da2683803/users/d13aa2d1db134687b8f7714009e6c679/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:08.278834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434f46-b6f6-4ac5-899e-772ed30c8a37 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:08.289222 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434f46-b6f6-4ac5-899e-772ed30c8a37 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:08.313869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:08.348211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcdf0c-5b5c-4ed6-a19f-f70b78932e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:08.348848 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcdf0c-5b5c-4ed6-a19f-f70b78932e92 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:08.349124 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcdf0c-5b5c-4ed6-a19f-f70b78932e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:08.349392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcdf0c-5b5c-4ed6-a19f-f70b78932e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:08.350100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bcdf0c-5b5c-4ed6-a19f-f70b78932e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:08.351680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bcdf0c-5b5c-4ed6-a19f-f70b78932e92 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:08.361579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1245/2486] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:08 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:08.365808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196fb11-9375-427b-90b0-d060e421c124 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:08.376293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196fb11-9375-427b-90b0-d060e421c124 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:08.380954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:08.381656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:08.381939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:08.382210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] PATH_INFO: `/v3/projects/95afef17853f4a2b9b8c659da2683803/users/d13aa2d1db134687b8f7714009e6c679/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:08.384163 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=95afef17853f4a2b9b8c659da2683803, user_id=d13aa2d1db134687b8f7714009e6c679, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:08.386875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:08.389061 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:08.399133 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1242/2487] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:08 2022] PUT /identity/v3/projects/95afef17853f4a2b9b8c659da2683803/users/d13aa2d1db134687b8f7714009e6c679/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:08.409868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17505a07-0559-4ed7-971c-f539965071cb None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:08.410153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17505a07-0559-4ed7-971c-f539965071cb None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:08.410428 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17505a07-0559-4ed7-971c-f539965071cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:08.437623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-17505a07-0559-4ed7-971c-f539965071cb None None] MFA Rules not processed for user `d13aa2d1db134687b8f7714009e6c679`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:08.438852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-17505a07-0559-4ed7-971c-f539965071cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:08.506138 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1246/2488] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:32:08.517848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:08.519561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:08.600016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94d3d-4488-4384-a137-325351bd84ac admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:08.603877 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e01e7f-0c04-42eb-8699-be8445e7de5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:08.604489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e01e7f-0c04-42eb-8699-be8445e7de5a service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:08.604768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e01e7f-0c04-42eb-8699-be8445e7de5a service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:08.605039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e01e7f-0c04-42eb-8699-be8445e7de5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:08.613976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e01e7f-0c04-42eb-8699-be8445e7de5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:08.615440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e01e7f-0c04-42eb-8699-be8445e7de5a service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:08.624584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1243/2489] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:09.945991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17505a07-0559-4ed7-971c-f539965071cb None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:10.008835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17505a07-0559-4ed7-971c-f539965071cb None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:10.013327 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:10.013989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:10.014268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:10.014538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] PATH_INFO: `/v3/users/7a4b0f60a44d4501a5d2e2279cc75311` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:10.015695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4b0f60a44d4501a5d2e2279cc75311)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:10.018600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:10.069084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4b0f60a44d4501a5d2e2279cc75311'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.071125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:10.077112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4b0f60a44d4501a5d2e2279cc75311'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.085494 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4b0f60a44d4501a5d2e2279cc75311'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.086276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4b0f60a44d4501a5d2e2279cc75311'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.098973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4b0f60a44d4501a5d2e2279cc75311'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.104833 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1247/2490] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:09 2022] DELETE /identity/v3/users/7a4b0f60a44d4501a5d2e2279cc75311 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:10.109392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a676f5-fd0b-4fb3-80de-668759ca1281 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:10.109965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a676f5-fd0b-4fb3-80de-668759ca1281 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:10.110096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a676f5-fd0b-4fb3-80de-668759ca1281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:10.129464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-46a676f5-fd0b-4fb3-80de-668759ca1281 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:10.130680 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-46a676f5-fd0b-4fb3-80de-668759ca1281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:10.189571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1244/2491] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:10.199308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:10.200699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:10.258365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a4961-9fb3-4c35-b3d6-8eec0858de3e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:10.262049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd2fa-5b00-4bb8-ae83-eeecc4e83deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:10.262669 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd2fa-5b00-4bb8-ae83-eeecc4e83deb service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:10.262947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd2fa-5b00-4bb8-ae83-eeecc4e83deb service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:10.263219 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd2fa-5b00-4bb8-ae83-eeecc4e83deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:10.272493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fd2fa-5b00-4bb8-ae83-eeecc4e83deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:10.273986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fd2fa-5b00-4bb8-ae83-eeecc4e83deb service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:10.283906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1248/2492] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:10.551929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a676f5-fd0b-4fb3-80de-668759ca1281 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:10.553554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-46a676f5-fd0b-4fb3-80de-668759ca1281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:10.566129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a676f5-fd0b-4fb3-80de-668759ca1281 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:10.572899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:10.573604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:10.573882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:10.574154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] PATH_INFO: `/v3/projects/6ce92c88991f4d7fbb66092136c2bbfd` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:10.575363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce92c88991f4d7fbb66092136c2bbfd)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:10.577274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:10.625616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce92c88991f4d7fbb66092136c2bbfd', 'request_id': 'req-097187c1-849f-427b-86f8-b635e62efaab'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.633598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce92c88991f4d7fbb66092136c2bbfd', 'request_id': 'req-097187c1-849f-427b-86f8-b635e62efaab'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.639501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce92c88991f4d7fbb66092136c2bbfd', 'request_id': 'req-097187c1-849f-427b-86f8-b635e62efaab'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.647831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:10.652516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] The token cache is being invalidate because project 6ce92c88991f4d7fbb66092136c2bbfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:10.653003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.654131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.655841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1245/2493] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:10 2022] DELETE /identity/v3/projects/6ce92c88991f4d7fbb66092136c2bbfd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:10.660476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd2fa-5b00-4bb8-ae83-eeecc4e83deb service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:10.670677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd2fa-5b00-4bb8-ae83-eeecc4e83deb service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:10.727288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:10.727924 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:10.728204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:10.728478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] PATH_INFO: `/v3/users/d13aa2d1db134687b8f7714009e6c679` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:10.729740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13aa2d1db134687b8f7714009e6c679)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:10.731741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:10.777936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13aa2d1db134687b8f7714009e6c679'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.785721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13aa2d1db134687b8f7714009e6c679'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.792997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13aa2d1db134687b8f7714009e6c679'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.793667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13aa2d1db134687b8f7714009e6c679'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.800825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:10.806946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-98152ed2-837e-4df6-adcf-c4c988089e63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13aa2d1db134687b8f7714009e6c679'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:10.813511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1249/2494] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:10 2022] DELETE /identity/v3/users/d13aa2d1db134687b8f7714009e6c679 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:11.093238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.094744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:11.110289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097187c1-849f-427b-86f8-b635e62efaab admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.170324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.170901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.171199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.171457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] PATH_INFO: `/v3/projects/95afef17853f4a2b9b8c659da2683803` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.172671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95afef17853f4a2b9b8c659da2683803)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.174798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.221499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95afef17853f4a2b9b8c659da2683803', 'request_id': 'req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:11.228773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95afef17853f4a2b9b8c659da2683803', 'request_id': 'req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:11.234333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95afef17853f4a2b9b8c659da2683803', 'request_id': 'req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:11.245778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] The token cache is being invalidate because project 95afef17853f4a2b9b8c659da2683803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:11.246070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:11.246715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-81e8f4f6-3816-4f1a-883d-b6eefed3a2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:11.248284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1246/2495] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:11 2022] DELETE /identity/v3/projects/95afef17853f4a2b9b8c659da2683803 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:11.253919 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067854f3-e5bb-4970-93d4-92a670755d18 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.254395 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067854f3-e5bb-4970-93d4-92a670755d18 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.254508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067854f3-e5bb-4970-93d4-92a670755d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.273783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-067854f3-e5bb-4970-93d4-92a670755d18 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:11.274995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-067854f3-e5bb-4970-93d4-92a670755d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:11.333810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1250/2496] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:11.342969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c61c-fb72-4df2-9c75-d7bbdf9e2eb3 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.343262 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c61c-fb72-4df2-9c75-d7bbdf9e2eb3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.343541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c61c-fb72-4df2-9c75-d7bbdf9e2eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.363205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-33e2c61c-fb72-4df2-9c75-d7bbdf9e2eb3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:11.364440 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-33e2c61c-fb72-4df2-9c75-d7bbdf9e2eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:11.372311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1247/2497] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:11.377019 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067854f3-e5bb-4970-93d4-92a670755d18 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.386556 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067854f3-e5bb-4970-93d4-92a670755d18 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.390993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.391644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.391920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.392190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.392877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:11.393203 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.394849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.406058 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1251/2498] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:32:11.410324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2c61c-fb72-4df2-9c75-d7bbdf9e2eb3 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.419487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2c61c-fb72-4df2-9c75-d7bbdf9e2eb3 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.424732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47854cb-f358-4f88-8ac6-e43c46f2603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.425283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47854cb-f358-4f88-8ac6-e43c46f2603c admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.425616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47854cb-f358-4f88-8ac6-e43c46f2603c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.425880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47854cb-f358-4f88-8ac6-e43c46f2603c admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.426536 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47854cb-f358-4f88-8ac6-e43c46f2603c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.428458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47854cb-f358-4f88-8ac6-e43c46f2603c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.455913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1248/2499] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:11 2022] POST /identity/v3/projects => generated 388 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:11.460046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.470569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0cc2a-4df3-4376-8786-51355b3340f0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.526459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f139d9-ace8-4761-add4-d7e3cf6b5da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.527077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f139d9-ace8-4761-add4-d7e3cf6b5da9 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.527360 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f139d9-ace8-4761-add4-d7e3cf6b5da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.527634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f139d9-ace8-4761-add4-d7e3cf6b5da9 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.528304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f139d9-ace8-4761-add4-d7e3cf6b5da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.530272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f139d9-ace8-4761-add4-d7e3cf6b5da9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.551665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1252/2500] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:11 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:11.555814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47854cb-f358-4f88-8ac6-e43c46f2603c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.566699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47854cb-f358-4f88-8ac6-e43c46f2603c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.571301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c2165-f93d-4c99-883c-8b8eabeb685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.571905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c2165-f93d-4c99-883c-8b8eabeb685e admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.572184 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c2165-f93d-4c99-883c-8b8eabeb685e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.572451 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c2165-f93d-4c99-883c-8b8eabeb685e admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.573167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c2165-f93d-4c99-883c-8b8eabeb685e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.574801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c2165-f93d-4c99-883c-8b8eabeb685e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.584717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1249/2501] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:11 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:11.588639 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f139d9-ace8-4761-add4-d7e3cf6b5da9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.598988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f139d9-ace8-4761-add4-d7e3cf6b5da9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.603726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0483e1-c0bf-4b0c-b8d3-ad9b8234e9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.604330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0483e1-c0bf-4b0c-b8d3-ad9b8234e9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.604610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0483e1-c0bf-4b0c-b8d3-ad9b8234e9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.604881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0483e1-c0bf-4b0c-b8d3-ad9b8234e9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.605633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0483e1-c0bf-4b0c-b8d3-ad9b8234e9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.607224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0483e1-c0bf-4b0c-b8d3-ad9b8234e9f3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.618306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1253/2502] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:11 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:11.622169 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c2165-f93d-4c99-883c-8b8eabeb685e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.632481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c2165-f93d-4c99-883c-8b8eabeb685e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.637067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bf690-c249-4200-ac37-c9efa9f0e342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.637754 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bf690-c249-4200-ac37-c9efa9f0e342 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.638035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bf690-c249-4200-ac37-c9efa9f0e342 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.638310 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bf690-c249-4200-ac37-c9efa9f0e342 admin admin] PATH_INFO: `/v3/projects/2358bc9503c0422db411a0ecabe4caac/users/a959db89bdb94f9691c4b4c7a48a526e/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.653086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6bf690-c249-4200-ac37-c9efa9f0e342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2358bc9503c0422db411a0ecabe4caac, user_id=a959db89bdb94f9691c4b4c7a48a526e, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.655768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6bf690-c249-4200-ac37-c9efa9f0e342 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.669144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1250/2503] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:11 2022] PUT /identity/v3/projects/2358bc9503c0422db411a0ecabe4caac/users/a959db89bdb94f9691c4b4c7a48a526e/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:11.680081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c5003-2ff6-47b1-b53b-2d5be116d1ec None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.680304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c5003-2ff6-47b1-b53b-2d5be116d1ec None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.680573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c5003-2ff6-47b1-b53b-2d5be116d1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.709866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3b3c5003-2ff6-47b1-b53b-2d5be116d1ec None None] MFA Rules not processed for user `a959db89bdb94f9691c4b4c7a48a526e`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:11.711057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3b3c5003-2ff6-47b1-b53b-2d5be116d1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:11.754120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:11.775965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1254/2504] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:11.780364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bf690-c249-4200-ac37-c9efa9f0e342 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.809482 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:11.844043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bf690-c249-4200-ac37-c9efa9f0e342 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.849037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efec4-b12a-49c6-a505-6275ab28a882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.849679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efec4-b12a-49c6-a505-6275ab28a882 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.849961 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efec4-b12a-49c6-a505-6275ab28a882 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.850234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efec4-b12a-49c6-a505-6275ab28a882 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.850903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594efec4-b12a-49c6-a505-6275ab28a882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.852817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594efec4-b12a-49c6-a505-6275ab28a882 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.879402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1251/2505] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:11 2022] POST /identity/v3/projects => generated 388 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:11.883445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c5003-2ff6-47b1-b53b-2d5be116d1ec None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.894647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c5003-2ff6-47b1-b53b-2d5be116d1ec None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.951975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04624c9-2350-4b01-aaf9-7ad67e74ebc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.952598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04624c9-2350-4b01-aaf9-7ad67e74ebc3 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.952986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04624c9-2350-4b01-aaf9-7ad67e74ebc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.953318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04624c9-2350-4b01-aaf9-7ad67e74ebc3 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.954784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04624c9-2350-4b01-aaf9-7ad67e74ebc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:11.956941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04624c9-2350-4b01-aaf9-7ad67e74ebc3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:11.980166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1255/2506] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:11 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:11.981459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efec4-b12a-49c6-a505-6275ab28a882 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:11.992351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efec4-b12a-49c6-a505-6275ab28a882 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:11.996970 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d5531-5412-4dec-9dcb-5417da89e26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:11.997610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d5531-5412-4dec-9dcb-5417da89e26d admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:11.997887 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d5531-5412-4dec-9dcb-5417da89e26d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:11.998156 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d5531-5412-4dec-9dcb-5417da89e26d admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:11.998870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d5531-5412-4dec-9dcb-5417da89e26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:12.001066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d5531-5412-4dec-9dcb-5417da89e26d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:12.011361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1252/2507] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:11 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:12.015399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04624c9-2350-4b01-aaf9-7ad67e74ebc3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:12.026367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04624c9-2350-4b01-aaf9-7ad67e74ebc3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:12.030995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26234ddb-13d6-44a1-aeb1-027af83a2c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:12.031651 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234ddb-13d6-44a1-aeb1-027af83a2c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:12.032273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234ddb-13d6-44a1-aeb1-027af83a2c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:12.032273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234ddb-13d6-44a1-aeb1-027af83a2c5d admin admin] PATH_INFO: `/v3/projects/490acbe048b845fcb44061ba6d6bd717/users/6a421817cf7a41208695029d4e76ad02/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:12.046710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26234ddb-13d6-44a1-aeb1-027af83a2c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=490acbe048b845fcb44061ba6d6bd717, user_id=6a421817cf7a41208695029d4e76ad02, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:12.050025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26234ddb-13d6-44a1-aeb1-027af83a2c5d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:12.062255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1256/2508] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:12 2022] PUT /identity/v3/projects/490acbe048b845fcb44061ba6d6bd717/users/6a421817cf7a41208695029d4e76ad02/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:32:12.066839 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d5531-5412-4dec-9dcb-5417da89e26d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:12.077003 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d5531-5412-4dec-9dcb-5417da89e26d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:12.134132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6cddf-f675-4ba5-873b-1ea04cf76e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:12.134747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6cddf-f675-4ba5-873b-1ea04cf76e22 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:12.135023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6cddf-f675-4ba5-873b-1ea04cf76e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:12.135291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6cddf-f675-4ba5-873b-1ea04cf76e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:12.135976 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6cddf-f675-4ba5-873b-1ea04cf76e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:12.137587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6cddf-f675-4ba5-873b-1ea04cf76e22 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:12.147706 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1253/2509] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:12 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:12.152031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26234ddb-13d6-44a1-aeb1-027af83a2c5d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:12.162156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26234ddb-13d6-44a1-aeb1-027af83a2c5d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:12.166726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:12.167370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:12.167644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:12.167910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] PATH_INFO: `/v3/projects/490acbe048b845fcb44061ba6d6bd717/users/6a421817cf7a41208695029d4e76ad02/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:12.169841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=490acbe048b845fcb44061ba6d6bd717, user_id=6a421817cf7a41208695029d4e76ad02, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:12.172927 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:12.182921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1257/2510] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:12 2022] PUT /identity/v3/projects/490acbe048b845fcb44061ba6d6bd717/users/6a421817cf7a41208695029d4e76ad02/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:12.193956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6818da-2260-4c27-a849-91aa1060c69d None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:12.194246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6818da-2260-4c27-a849-91aa1060c69d None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:12.194551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6818da-2260-4c27-a849-91aa1060c69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:12.222899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7d6818da-2260-4c27-a849-91aa1060c69d None None] MFA Rules not processed for user `6a421817cf7a41208695029d4e76ad02`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:12.224031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7d6818da-2260-4c27-a849-91aa1060c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:12.292611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1254/2511] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:12 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:12.304432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:12.306329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:12.363768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefb5c6-b93e-4496-9704-2586b67df1ac admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:12.367453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002b83e-2884-4a2a-a444-ab22e1c151b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:12.368059 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002b83e-2884-4a2a-a444-ab22e1c151b1 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:12.368341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002b83e-2884-4a2a-a444-ab22e1c151b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:12.368615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002b83e-2884-4a2a-a444-ab22e1c151b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:12.378245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002b83e-2884-4a2a-a444-ab22e1c151b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:12.379668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002b83e-2884-4a2a-a444-ab22e1c151b1 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:12.389023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1258/2512] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:12 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:12.454747 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6818da-2260-4c27-a849-91aa1060c69d None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:12.479730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:12.518020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6818da-2260-4c27-a849-91aa1060c69d None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:12.523386 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:12.523912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:12.524189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:12.524458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] PATH_INFO: `/v3/users/a959db89bdb94f9691c4b4c7a48a526e` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:12.525689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a959db89bdb94f9691c4b4c7a48a526e)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:12.527674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:12.572921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a959db89bdb94f9691c4b4c7a48a526e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:12.580741 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a959db89bdb94f9691c4b4c7a48a526e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:12.581383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a959db89bdb94f9691c4b4c7a48a526e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:12.594249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a959db89bdb94f9691c4b4c7a48a526e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:12.594895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a959db89bdb94f9691c4b4c7a48a526e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:12.606650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1255/2513] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:12 2022] DELETE /identity/v3/users/a959db89bdb94f9691c4b4c7a48a526e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:12.611153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52688-0f5e-4e77-b684-b43e3067b182 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:12.611444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52688-0f5e-4e77-b684-b43e3067b182 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:12.611833 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52688-0f5e-4e77-b684-b43e3067b182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:12.631740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-91d52688-0f5e-4e77-b684-b43e3067b182 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:12.632965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-91d52688-0f5e-4e77-b684-b43e3067b182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:12.665989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:12.691916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1259/2514] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:12.705896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:12.707303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:12.765771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cf25b-3efc-462f-aa36-b22d89a73bab admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:12.770506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cfc1b-fe37-498e-9c5d-10fc2cad076d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:12.771126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cfc1b-fe37-498e-9c5d-10fc2cad076d service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:12.771405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cfc1b-fe37-498e-9c5d-10fc2cad076d service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:12.771682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cfc1b-fe37-498e-9c5d-10fc2cad076d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:12.780863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cfc1b-fe37-498e-9c5d-10fc2cad076d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:12.782431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cfc1b-fe37-498e-9c5d-10fc2cad076d service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:12.792142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1256/2515] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:32:13.054885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d52688-0f5e-4e77-b684-b43e3067b182 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:13.056566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-91d52688-0f5e-4e77-b684-b43e3067b182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:13.069741 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d52688-0f5e-4e77-b684-b43e3067b182 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:13.074475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:13.075509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:13.075509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:13.075835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] PATH_INFO: `/v3/projects/2358bc9503c0422db411a0ecabe4caac` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:13.076979 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2358bc9503c0422db411a0ecabe4caac)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:13.078967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:13.126765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2358bc9503c0422db411a0ecabe4caac', 'request_id': 'req-e7af9209-d064-46d4-88c2-efab676d38c4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.132098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2358bc9503c0422db411a0ecabe4caac', 'request_id': 'req-e7af9209-d064-46d4-88c2-efab676d38c4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.139491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2358bc9503c0422db411a0ecabe4caac', 'request_id': 'req-e7af9209-d064-46d4-88c2-efab676d38c4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.150849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] The token cache is being invalidate because project 2358bc9503c0422db411a0ecabe4caac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:13.151133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.151750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.153252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1260/2516] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:13 2022] DELETE /identity/v3/projects/2358bc9503c0422db411a0ecabe4caac => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:13.157767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cfc1b-fe37-498e-9c5d-10fc2cad076d service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:13.167507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cfc1b-fe37-498e-9c5d-10fc2cad076d service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:13.208555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:13.225649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:13.226283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:13.226560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:13.226830 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] PATH_INFO: `/v3/users/6a421817cf7a41208695029d4e76ad02` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:13.227986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a421817cf7a41208695029d4e76ad02)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:13.229989 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:13.274722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a421817cf7a41208695029d4e76ad02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.283222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a421817cf7a41208695029d4e76ad02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.283912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a421817cf7a41208695029d4e76ad02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.295487 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a421817cf7a41208695029d4e76ad02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.296137 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-d4db9ced-1bf5-45fc-b7ab-33247670a89a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a421817cf7a41208695029d4e76ad02'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.308891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1257/2517] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:13 2022] DELETE /identity/v3/users/6a421817cf7a41208695029d4e76ad02 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:13.583213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:13.584661 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:13.603051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af9209-d064-46d4-88c2-efab676d38c4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:13.659558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:13.660204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:13.660486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:13.660757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] PATH_INFO: `/v3/projects/490acbe048b845fcb44061ba6d6bd717` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:13.661942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490acbe048b845fcb44061ba6d6bd717)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:13.663862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:13.710824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490acbe048b845fcb44061ba6d6bd717', 'request_id': 'req-023e9b2c-2d3f-462c-b68c-59a8a9f62125'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.716236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490acbe048b845fcb44061ba6d6bd717', 'request_id': 'req-023e9b2c-2d3f-462c-b68c-59a8a9f62125'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.724025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490acbe048b845fcb44061ba6d6bd717', 'request_id': 'req-023e9b2c-2d3f-462c-b68c-59a8a9f62125'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.725249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:13.738133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] The token cache is being invalidate because project 490acbe048b845fcb44061ba6d6bd717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:13.738424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.739057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-023e9b2c-2d3f-462c-b68c-59a8a9f62125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:13.740562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1261/2518] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:13 2022] DELETE /identity/v3/projects/490acbe048b845fcb44061ba6d6bd717 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:32:13.745697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9fbe0-caf7-44c5-8373-9609d08c1087 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:13.745985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9fbe0-caf7-44c5-8373-9609d08c1087 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:13.746258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9fbe0-caf7-44c5-8373-9609d08c1087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:13.765426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-25c9fbe0-caf7-44c5-8373-9609d08c1087 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:13.767308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-25c9fbe0-caf7-44c5-8373-9609d08c1087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:13.827292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1258/2519] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:32:13.839021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f39f5-f663-4a57-9add-53b9ccbd265c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:13.839311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f39f5-f663-4a57-9add-53b9ccbd265c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:13.839585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f39f5-f663-4a57-9add-53b9ccbd265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:13.860707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e89f39f5-f663-4a57-9add-53b9ccbd265c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:13.861949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e89f39f5-f663-4a57-9add-53b9ccbd265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:13.870065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1262/2520] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:13.874316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9fbe0-caf7-44c5-8373-9609d08c1087 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:13.884771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9fbe0-caf7-44c5-8373-9609d08c1087 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:13.889334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:13.889962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:13.890241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:13.890511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:13.891773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:13.892066 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:13.893682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:13.904963 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1259/2521] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:32:13.909367 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f39f5-f663-4a57-9add-53b9ccbd265c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:13.918641 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f39f5-f663-4a57-9add-53b9ccbd265c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:13.923340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc176b6-1bf2-4ac2-8abb-fa5c4df4f2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:13.923939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc176b6-1bf2-4ac2-8abb-fa5c4df4f2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:13.924219 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc176b6-1bf2-4ac2-8abb-fa5c4df4f2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:13.924492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc176b6-1bf2-4ac2-8abb-fa5c4df4f2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:13.925159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc176b6-1bf2-4ac2-8abb-fa5c4df4f2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:13.927132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc176b6-1bf2-4ac2-8abb-fa5c4df4f2e2 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:13.960782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1263/2522] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:13 2022] POST /identity/v3/projects => generated 388 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:13.962087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:13.972295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc6721-df61-4217-8088-8827991a6fa0 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.018398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:14.029739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be455d-740f-459d-b2f3-42c92f3a9686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.030378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be455d-740f-459d-b2f3-42c92f3a9686 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.030661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be455d-740f-459d-b2f3-42c92f3a9686 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.030938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be455d-740f-459d-b2f3-42c92f3a9686 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.031592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be455d-740f-459d-b2f3-42c92f3a9686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.033561 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be455d-740f-459d-b2f3-42c92f3a9686 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.054562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1260/2523] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:13 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:14.058858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc176b6-1bf2-4ac2-8abb-fa5c4df4f2e2 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.069701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc176b6-1bf2-4ac2-8abb-fa5c4df4f2e2 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.074378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bc8ad-fbf6-48e1-bd78-3328a4c591e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.074984 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bc8ad-fbf6-48e1-bd78-3328a4c591e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.075262 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bc8ad-fbf6-48e1-bd78-3328a4c591e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.075534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bc8ad-fbf6-48e1-bd78-3328a4c591e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.076220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bc8ad-fbf6-48e1-bd78-3328a4c591e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.077915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bc8ad-fbf6-48e1-bd78-3328a4c591e9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.087920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1264/2524] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:14 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:14.091658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be455d-740f-459d-b2f3-42c92f3a9686 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.102371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be455d-740f-459d-b2f3-42c92f3a9686 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.107191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426109e-ca38-4a8a-9504-ee972a3d0733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.107808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426109e-ca38-4a8a-9504-ee972a3d0733 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.108116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426109e-ca38-4a8a-9504-ee972a3d0733 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.108389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426109e-ca38-4a8a-9504-ee972a3d0733 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.109087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426109e-ca38-4a8a-9504-ee972a3d0733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.111746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426109e-ca38-4a8a-9504-ee972a3d0733 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.125517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1261/2525] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:14 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:14.130215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bc8ad-fbf6-48e1-bd78-3328a4c591e9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.140221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bc8ad-fbf6-48e1-bd78-3328a4c591e9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.145164 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3ca92-3e5d-4d4a-94c9-b2c6f85764b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.145836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3ca92-3e5d-4d4a-94c9-b2c6f85764b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.146113 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3ca92-3e5d-4d4a-94c9-b2c6f85764b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.146382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3ca92-3e5d-4d4a-94c9-b2c6f85764b2 admin admin] PATH_INFO: `/v3/projects/f51412d188f3458eac5a4886ddbe0f7a/users/8228d2a77a8743ea924dae7b2ed58e3e/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.161837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3ca92-3e5d-4d4a-94c9-b2c6f85764b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51412d188f3458eac5a4886ddbe0f7a, user_id=8228d2a77a8743ea924dae7b2ed58e3e, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.164498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3ca92-3e5d-4d4a-94c9-b2c6f85764b2 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.177685 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1265/2526] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:14 2022] PUT /identity/v3/projects/f51412d188f3458eac5a4886ddbe0f7a/users/8228d2a77a8743ea924dae7b2ed58e3e/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:32:14.187853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7fb8-5cc0-4566-9138-c7962ce76093 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.188370 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7fb8-5cc0-4566-9138-c7962ce76093 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.188490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7fb8-5cc0-4566-9138-c7962ce76093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.216576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6a2c7fb8-5cc0-4566-9138-c7962ce76093 None None] MFA Rules not processed for user `8228d2a77a8743ea924dae7b2ed58e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:14.218125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6a2c7fb8-5cc0-4566-9138-c7962ce76093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:14.282296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1262/2527] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:14 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:14.286441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3ca92-3e5d-4d4a-94c9-b2c6f85764b2 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.347665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3ca92-3e5d-4d4a-94c9-b2c6f85764b2 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.352363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b824-b3f0-4816-bbad-f3d4b8622a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.352970 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b824-b3f0-4816-bbad-f3d4b8622a89 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.353250 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b824-b3f0-4816-bbad-f3d4b8622a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.353568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b824-b3f0-4816-bbad-f3d4b8622a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.354221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5b824-b3f0-4816-bbad-f3d4b8622a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.356119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5b824-b3f0-4816-bbad-f3d4b8622a89 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.382491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1266/2528] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:14 2022] POST /identity/v3/projects => generated 388 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:14.386558 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c7fb8-5cc0-4566-9138-c7962ce76093 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.398023 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c7fb8-5cc0-4566-9138-c7962ce76093 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.459786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915f897-323e-432f-b6fe-a1224a4e353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.460415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f897-323e-432f-b6fe-a1224a4e353b admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.460701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f897-323e-432f-b6fe-a1224a4e353b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.460979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f897-323e-432f-b6fe-a1224a4e353b admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.461678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915f897-323e-432f-b6fe-a1224a4e353b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.463632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915f897-323e-432f-b6fe-a1224a4e353b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.486006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1263/2529] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:14 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:14.489992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b824-b3f0-4816-bbad-f3d4b8622a89 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.500159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b824-b3f0-4816-bbad-f3d4b8622a89 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.505468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4012039-3361-4581-b6db-a6401524e9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.506075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4012039-3361-4581-b6db-a6401524e9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.506354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4012039-3361-4581-b6db-a6401524e9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.506645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4012039-3361-4581-b6db-a6401524e9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.507331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4012039-3361-4581-b6db-a6401524e9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.508908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4012039-3361-4581-b6db-a6401524e9d4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.519455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1267/2530] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:14 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:14.523474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915f897-323e-432f-b6fe-a1224a4e353b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.533842 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915f897-323e-432f-b6fe-a1224a4e353b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.538503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309970e6-a3a8-431d-8df7-3ce564e56777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.539237 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309970e6-a3a8-431d-8df7-3ce564e56777 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.539527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309970e6-a3a8-431d-8df7-3ce564e56777 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.539808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309970e6-a3a8-431d-8df7-3ce564e56777 admin admin] PATH_INFO: `/v3/projects/d71a62b862584a4bbd1683544795ff32/users/820d1502582d489da15846b65c75a2e4/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.555081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309970e6-a3a8-431d-8df7-3ce564e56777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71a62b862584a4bbd1683544795ff32, user_id=820d1502582d489da15846b65c75a2e4, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.557845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309970e6-a3a8-431d-8df7-3ce564e56777 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.570784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1264/2531] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:14 2022] PUT /identity/v3/projects/d71a62b862584a4bbd1683544795ff32/users/820d1502582d489da15846b65c75a2e4/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:14.574703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4012039-3361-4581-b6db-a6401524e9d4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.584754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4012039-3361-4581-b6db-a6401524e9d4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.646394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d744ba-c47b-4a66-b96e-951cf24b7200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.646932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d744ba-c47b-4a66-b96e-951cf24b7200 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.647209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d744ba-c47b-4a66-b96e-951cf24b7200 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.647478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d744ba-c47b-4a66-b96e-951cf24b7200 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.648159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d744ba-c47b-4a66-b96e-951cf24b7200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.649784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d744ba-c47b-4a66-b96e-951cf24b7200 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.654786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:14.664228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1268/2532] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:14 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:14.665578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309970e6-a3a8-431d-8df7-3ce564e56777 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.676889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309970e6-a3a8-431d-8df7-3ce564e56777 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.681505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.682161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.682437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.682707 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] PATH_INFO: `/v3/projects/d71a62b862584a4bbd1683544795ff32/users/820d1502582d489da15846b65c75a2e4/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.684679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71a62b862584a4bbd1683544795ff32, user_id=820d1502582d489da15846b65c75a2e4, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.687344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.697429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1265/2533] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:14 2022] PUT /identity/v3/projects/d71a62b862584a4bbd1683544795ff32/users/820d1502582d489da15846b65c75a2e4/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:14.707764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16aff91-5aba-4218-80fb-f980d4d3804f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.708055 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16aff91-5aba-4218-80fb-f980d4d3804f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.708334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16aff91-5aba-4218-80fb-f980d4d3804f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.740002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f16aff91-5aba-4218-80fb-f980d4d3804f None None] MFA Rules not processed for user `820d1502582d489da15846b65c75a2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:14.741272 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f16aff91-5aba-4218-80fb-f980d4d3804f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:14.807898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1269/2534] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:14 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:14.818413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:14.820115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:14.835939 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:14.880778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e74b0-26b0-4d3f-bf35-42ae8270e62c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:14.884465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e20d29-816e-45d9-bc90-6533957c9038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:14.885074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e20d29-816e-45d9-bc90-6533957c9038 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:14.885353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e20d29-816e-45d9-bc90-6533957c9038 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:14.885661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e20d29-816e-45d9-bc90-6533957c9038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:14.895108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e20d29-816e-45d9-bc90-6533957c9038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:14.896589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e20d29-816e-45d9-bc90-6533957c9038 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:14.905841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1266/2535] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:14 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:15.452966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16aff91-5aba-4218-80fb-f980d4d3804f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:15.514317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16aff91-5aba-4218-80fb-f980d4d3804f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:15.519620 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:15.520303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:15.520591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:15.520866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] PATH_INFO: `/v3/users/8228d2a77a8743ea924dae7b2ed58e3e` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:15.522035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8228d2a77a8743ea924dae7b2ed58e3e)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:15.523950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:15.569184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8228d2a77a8743ea924dae7b2ed58e3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:15.576281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8228d2a77a8743ea924dae7b2ed58e3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:15.584148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8228d2a77a8743ea924dae7b2ed58e3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:15.584771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8228d2a77a8743ea924dae7b2ed58e3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:15.596896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8228d2a77a8743ea924dae7b2ed58e3e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:15.602631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1270/2536] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:15 2022] DELETE /identity/v3/users/8228d2a77a8743ea924dae7b2ed58e3e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:15.607097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0992c5-5978-4892-b7dd-5e05379bf765 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:15.607550 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0992c5-5978-4892-b7dd-5e05379bf765 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:15.607669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0992c5-5978-4892-b7dd-5e05379bf765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:15.626094 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9e0992c5-5978-4892-b7dd-5e05379bf765 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:15.627460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9e0992c5-5978-4892-b7dd-5e05379bf765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:15.684172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1267/2537] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:15.694679 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:15.696131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:15.702652 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:15.755667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866c8b5-e506-48ea-ab87-d01fc162d60b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:15.759351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d8473-8863-486c-b451-538fd9bec366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:15.759965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8473-8863-486c-b451-538fd9bec366 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:15.760246 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8473-8863-486c-b451-538fd9bec366 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:15.760519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8473-8863-486c-b451-538fd9bec366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:15.770132 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d8473-8863-486c-b451-538fd9bec366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:15.771563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d8473-8863-486c-b451-538fd9bec366 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:15.781014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1271/2538] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:16.054189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0992c5-5978-4892-b7dd-5e05379bf765 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:16.055687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9e0992c5-5978-4892-b7dd-5e05379bf765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:16.068448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0992c5-5978-4892-b7dd-5e05379bf765 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:16.073069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:16.073756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:16.074040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:16.074317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] PATH_INFO: `/v3/projects/f51412d188f3458eac5a4886ddbe0f7a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:16.075489 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] RBAC: Authorizing `identity:delete_project(project_id=f51412d188f3458eac5a4886ddbe0f7a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:16.077439 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:16.126831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f51412d188f3458eac5a4886ddbe0f7a', 'request_id': 'req-e195583c-9678-4e50-bc8f-3379ffdbb6da'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.134080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f51412d188f3458eac5a4886ddbe0f7a', 'request_id': 'req-e195583c-9678-4e50-bc8f-3379ffdbb6da'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.139910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f51412d188f3458eac5a4886ddbe0f7a', 'request_id': 'req-e195583c-9678-4e50-bc8f-3379ffdbb6da'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.151102 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] The token cache is being invalidate because project f51412d188f3458eac5a4886ddbe0f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:16.151387 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.152021 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.153578 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1268/2539] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:16 2022] DELETE /identity/v3/projects/f51412d188f3458eac5a4886ddbe0f7a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:16.158095 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d8473-8863-486c-b451-538fd9bec366 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:16.167677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d8473-8863-486c-b451-538fd9bec366 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:16.223009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:16.223578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:16.223860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:16.224135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] PATH_INFO: `/v3/users/820d1502582d489da15846b65c75a2e4` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:16.225298 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820d1502582d489da15846b65c75a2e4)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:16.227277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:16.276289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820d1502582d489da15846b65c75a2e4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.283041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820d1502582d489da15846b65c75a2e4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.289796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820d1502582d489da15846b65c75a2e4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.290504 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820d1502582d489da15846b65c75a2e4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.303094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a6ef1db6-2461-47c6-acc8-b83d4d360e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820d1502582d489da15846b65c75a2e4'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.308911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1272/2540] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:16 2022] DELETE /identity/v3/users/820d1502582d489da15846b65c75a2e4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:16.586960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:16.588369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:16.604920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195583c-9678-4e50-bc8f-3379ffdbb6da admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:16.640030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:16.666443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:16.667085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:16.667368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:16.667644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] PATH_INFO: `/v3/projects/d71a62b862584a4bbd1683544795ff32` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:16.668815 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71a62b862584a4bbd1683544795ff32)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:16.670778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:16.716442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71a62b862584a4bbd1683544795ff32', 'request_id': 'req-6d7db5da-addf-463d-b880-ece2d88fa140'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.723253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71a62b862584a4bbd1683544795ff32', 'request_id': 'req-6d7db5da-addf-463d-b880-ece2d88fa140'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.729018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71a62b862584a4bbd1683544795ff32', 'request_id': 'req-6d7db5da-addf-463d-b880-ece2d88fa140'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.740442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] The token cache is being invalidate because project d71a62b862584a4bbd1683544795ff32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:16.740727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.741364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-6d7db5da-addf-463d-b880-ece2d88fa140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:16.742912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1269/2541] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:16 2022] DELETE /identity/v3/projects/d71a62b862584a4bbd1683544795ff32 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:16.748078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625e67-9fb2-417e-b6a3-13cefd22314c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:16.748576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625e67-9fb2-417e-b6a3-13cefd22314c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:16.748711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625e67-9fb2-417e-b6a3-13cefd22314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:16.770265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-41625e67-9fb2-417e-b6a3-13cefd22314c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:16.771471 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-41625e67-9fb2-417e-b6a3-13cefd22314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:16.791409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:16.833213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1273/2542] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:16.841299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae0c61-3c0a-4d39-85d4-cf3cc4aaa667 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:16.841732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae0c61-3c0a-4d39-85d4-cf3cc4aaa667 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:16.842017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae0c61-3c0a-4d39-85d4-cf3cc4aaa667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:16.862646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8bae0c61-3c0a-4d39-85d4-cf3cc4aaa667 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:16.863857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8bae0c61-3c0a-4d39-85d4-cf3cc4aaa667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:16.871982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1270/2543] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:16.877062 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41625e67-9fb2-417e-b6a3-13cefd22314c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:16.886683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41625e67-9fb2-417e-b6a3-13cefd22314c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:16.891469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:16.892066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:16.892341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:16.892607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:16.893283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:16.893619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:16.895198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:16.907689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1274/2544] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:32:16.911769 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae0c61-3c0a-4d39-85d4-cf3cc4aaa667 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:16.921166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae0c61-3c0a-4d39-85d4-cf3cc4aaa667 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:16.926790 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1efad1-2bd9-4eb1-89e7-5e23498be80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:16.927394 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1efad1-2bd9-4eb1-89e7-5e23498be80a admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:16.927705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1efad1-2bd9-4eb1-89e7-5e23498be80a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:16.927983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1efad1-2bd9-4eb1-89e7-5e23498be80a admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:16.928649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1efad1-2bd9-4eb1-89e7-5e23498be80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:16.930583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1efad1-2bd9-4eb1-89e7-5e23498be80a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:16.960323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1271/2545] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:16 2022] POST /identity/v3/projects => generated 372 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:16.961616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:16.972343 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6d83b-48a9-46f3-8c49-1cc3dab5c22f admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:17.027120 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55ec80-f176-46d7-b606-306ad826e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:17.027663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ec80-f176-46d7-b606-306ad826e2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.027944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ec80-f176-46d7-b606-306ad826e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.028220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ec80-f176-46d7-b606-306ad826e2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.028873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55ec80-f176-46d7-b606-306ad826e2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:17.030818 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55ec80-f176-46d7-b606-306ad826e2f1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:17.055181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1275/2546] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:16 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:17.056450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1efad1-2bd9-4eb1-89e7-5e23498be80a admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:17.067041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1efad1-2bd9-4eb1-89e7-5e23498be80a admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:17.071704 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cc0c3-f922-49fe-9331-e076a4d10dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:17.072313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cc0c3-f922-49fe-9331-e076a4d10dcb admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.072597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cc0c3-f922-49fe-9331-e076a4d10dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.072873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cc0c3-f922-49fe-9331-e076a4d10dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.073600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98cc0c3-f922-49fe-9331-e076a4d10dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:17.075200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98cc0c3-f922-49fe-9331-e076a4d10dcb admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:17.087506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1272/2547] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:17 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:17.091707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55ec80-f176-46d7-b606-306ad826e2f1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:17.102404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55ec80-f176-46d7-b606-306ad826e2f1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:17.107064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a875ac3-5831-4547-99f2-aeb2ddf0ed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:17.107676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a875ac3-5831-4547-99f2-aeb2ddf0ed89 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.107974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a875ac3-5831-4547-99f2-aeb2ddf0ed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.108221 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a875ac3-5831-4547-99f2-aeb2ddf0ed89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.108932 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a875ac3-5831-4547-99f2-aeb2ddf0ed89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:17.111208 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a875ac3-5831-4547-99f2-aeb2ddf0ed89 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:17.122029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1276/2548] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:17 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:17.126342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cc0c3-f922-49fe-9331-e076a4d10dcb admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:17.136688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cc0c3-f922-49fe-9331-e076a4d10dcb admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:17.141429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:17.142131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.142479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.142760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] PATH_INFO: `/v3/projects/b8408be2dda54b699d66bcc63b6c9057/users/61546429930c439ab47cca82b5d84e94/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.158122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8408be2dda54b699d66bcc63b6c9057, user_id=61546429930c439ab47cca82b5d84e94, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:17.160795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:17.174940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1273/2549] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:17 2022] PUT /identity/v3/projects/b8408be2dda54b699d66bcc63b6c9057/users/61546429930c439ab47cca82b5d84e94/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:17.185248 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f369-6aa1-4bd7-8850-cf425c8bcc2e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.185575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f369-6aa1-4bd7-8850-cf425c8bcc2e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.185852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f369-6aa1-4bd7-8850-cf425c8bcc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.212914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f798f369-6aa1-4bd7-8850-cf425c8bcc2e None None] MFA Rules not processed for user `61546429930c439ab47cca82b5d84e94`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:17.214148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f798f369-6aa1-4bd7-8850-cf425c8bcc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:17.278269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1277/2550] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:17.289201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:17.290971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:17.351369 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ffa8-ffaf-42f5-a2e6-38f20a961814 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:17.355044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530a561-30f8-4d7f-819d-d07da88c053c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:17.355661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530a561-30f8-4d7f-819d-d07da88c053c service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.355935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530a561-30f8-4d7f-819d-d07da88c053c service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.356206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530a561-30f8-4d7f-819d-d07da88c053c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.365674 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0530a561-30f8-4d7f-819d-d07da88c053c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:17.367111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0530a561-30f8-4d7f-819d-d07da88c053c service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:17.376940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1274/2551] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:17.397475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798f369-6aa1-4bd7-8850-cf425c8bcc2e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:17.423554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:17.463950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798f369-6aa1-4bd7-8850-cf425c8bcc2e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:17.469283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:17.470255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.470587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.470978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] PATH_INFO: `/v3/users/61546429930c439ab47cca82b5d84e94` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.472486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61546429930c439ab47cca82b5d84e94)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:17.474701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:17.527071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61546429930c439ab47cca82b5d84e94'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:17.534963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61546429930c439ab47cca82b5d84e94'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:17.543766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61546429930c439ab47cca82b5d84e94'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:17.544539 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61546429930c439ab47cca82b5d84e94'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:17.575024 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61546429930c439ab47cca82b5d84e94'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:17.582290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1278/2552] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:17 2022] DELETE /identity/v3/users/61546429930c439ab47cca82b5d84e94 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:17.586890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978713b-b504-47b6-a105-0603dd8e60c4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.587331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978713b-b504-47b6-a105-0603dd8e60c4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.587687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978713b-b504-47b6-a105-0603dd8e60c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.603022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:17.620494 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-2978713b-b504-47b6-a105-0603dd8e60c4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:17.621784 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2978713b-b504-47b6-a105-0603dd8e60c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:17.684568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1275/2553] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:17.694945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:17.696328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:17.752565 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5a85c-667d-45b5-9d63-5d60f54fe614 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:17.756228 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba7cd2-ffb9-4d9d-a73e-05c0d2309faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:17.756839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba7cd2-ffb9-4d9d-a73e-05c0d2309faf service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:17.757127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba7cd2-ffb9-4d9d-a73e-05c0d2309faf service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:17.757388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba7cd2-ffb9-4d9d-a73e-05c0d2309faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:17.767463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba7cd2-ffb9-4d9d-a73e-05c0d2309faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:17.768887 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba7cd2-ffb9-4d9d-a73e-05c0d2309faf service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:17.778232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1279/2554] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:32:18.044513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978713b-b504-47b6-a105-0603dd8e60c4 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:18.045958 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2978713b-b504-47b6-a105-0603dd8e60c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:18.060247 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978713b-b504-47b6-a105-0603dd8e60c4 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:18.065901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:18.066560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.066880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.067199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] PATH_INFO: `/v3/projects/b8408be2dda54b699d66bcc63b6c9057` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.068404 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8408be2dda54b699d66bcc63b6c9057)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:18.070418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:18.117600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8408be2dda54b699d66bcc63b6c9057', 'request_id': 'req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:18.125607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8408be2dda54b699d66bcc63b6c9057', 'request_id': 'req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:18.131127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8408be2dda54b699d66bcc63b6c9057', 'request_id': 'req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:18.142152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] The token cache is being invalidate because project b8408be2dda54b699d66bcc63b6c9057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:18.142438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:18.143069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-3e1684db-bc03-43a5-97d9-39ba87a8e8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:18.144580 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1276/2555] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:18 2022] DELETE /identity/v3/projects/b8408be2dda54b699d66bcc63b6c9057 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:18.149427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915a7f-6913-4285-8d23-c4c1f7eddfee None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.149751 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915a7f-6913-4285-8d23-c4c1f7eddfee None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.150028 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915a7f-6913-4285-8d23-c4c1f7eddfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.169165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-92915a7f-6913-4285-8d23-c4c1f7eddfee None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:18.170406 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-92915a7f-6913-4285-8d23-c4c1f7eddfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:18.211032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:18.231534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1280/2556] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:18.240780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb2b2-4437-4c59-96c6-be242afb225c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.241070 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb2b2-4437-4c59-96c6-be242afb225c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.241349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb2b2-4437-4c59-96c6-be242afb225c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.261226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-affbb2b2-4437-4c59-96c6-be242afb225c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:18.262500 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-affbb2b2-4437-4c59-96c6-be242afb225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:18.270555 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1277/2557] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:18.274702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915a7f-6913-4285-8d23-c4c1f7eddfee None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:18.284385 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915a7f-6913-4285-8d23-c4c1f7eddfee None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:18.288950 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:18.289600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.289883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.290153 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.290839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:18.291142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:18.292729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:18.304696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1281/2558] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:32:18.308780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbb2b2-4437-4c59-96c6-be242afb225c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:18.318667 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbb2b2-4437-4c59-96c6-be242afb225c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:18.323308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3258e-4b31-4956-a6ad-b98bd23646e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:18.323916 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3258e-4b31-4956-a6ad-b98bd23646e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.324196 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3258e-4b31-4956-a6ad-b98bd23646e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.324486 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3258e-4b31-4956-a6ad-b98bd23646e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.325130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3258e-4b31-4956-a6ad-b98bd23646e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:18.327080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3258e-4b31-4956-a6ad-b98bd23646e3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:18.354038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1278/2559] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:18 2022] POST /identity/v3/projects => generated 376 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:18.358210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:18.370672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398758d-2fbe-4715-87f6-e3da63b26bc4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:18.435465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026d5a7-6547-4c49-96a4-03baae31a739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:18.436096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d5a7-6547-4c49-96a4-03baae31a739 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.436387 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d5a7-6547-4c49-96a4-03baae31a739 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.436690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d5a7-6547-4c49-96a4-03baae31a739 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.437474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026d5a7-6547-4c49-96a4-03baae31a739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:18.439491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026d5a7-6547-4c49-96a4-03baae31a739 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:18.461348 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1282/2560] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:18 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:18.465892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3258e-4b31-4956-a6ad-b98bd23646e3 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:18.477895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3258e-4b31-4956-a6ad-b98bd23646e3 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:18.483254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69259bae-6207-4a5e-8ce9-c198e89a234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:18.483882 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69259bae-6207-4a5e-8ce9-c198e89a234d admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.484162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69259bae-6207-4a5e-8ce9-c198e89a234d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.485564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69259bae-6207-4a5e-8ce9-c198e89a234d admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.486267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69259bae-6207-4a5e-8ce9-c198e89a234d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:18.487880 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69259bae-6207-4a5e-8ce9-c198e89a234d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:18.497737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1279/2561] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:18 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:18.501721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026d5a7-6547-4c49-96a4-03baae31a739 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:18.511574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026d5a7-6547-4c49-96a4-03baae31a739 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:18.516657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08c21a-02e1-4fe7-a839-928fd048625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:18.517257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c21a-02e1-4fe7-a839-928fd048625d admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.517574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c21a-02e1-4fe7-a839-928fd048625d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.517846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c21a-02e1-4fe7-a839-928fd048625d admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.518560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08c21a-02e1-4fe7-a839-928fd048625d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:18.520172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08c21a-02e1-4fe7-a839-928fd048625d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:18.532724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1283/2562] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:18 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:18.537217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69259bae-6207-4a5e-8ce9-c198e89a234d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:18.547838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69259bae-6207-4a5e-8ce9-c198e89a234d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:18.552420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:18.553071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.553347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.553651 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] PATH_INFO: `/v3/projects/6a0a0b446c6b42c8927636a6357b3456/users/724450d4209b43d5ac95d534e78c68a2/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.569141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a0a0b446c6b42c8927636a6357b3456, user_id=724450d4209b43d5ac95d534e78c68a2, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:18.571962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:18.584444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1280/2563] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:18 2022] PUT /identity/v3/projects/6a0a0b446c6b42c8927636a6357b3456/users/724450d4209b43d5ac95d534e78c68a2/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:32:18.652821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba403a-cdc0-4d0b-a8b4-73b9aa12e56a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.653091 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba403a-cdc0-4d0b-a8b4-73b9aa12e56a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.653371 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba403a-cdc0-4d0b-a8b4-73b9aa12e56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.686770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-bcba403a-cdc0-4d0b-a8b4-73b9aa12e56a None None] MFA Rules not processed for user `724450d4209b43d5ac95d534e78c68a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:18.688107 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bcba403a-cdc0-4d0b-a8b4-73b9aa12e56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:18.761673 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1284/2564] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:18.775777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:18.777530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:18.814339 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:18.865016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc4574-0d1f-4349-b4a4-96779e507be0 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:18.869129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33183e28de8f487f82f92f0151e7c73c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:18.869804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:18.870087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:18.870364 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:18.880401 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:18.881884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:18.892595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1281/2565] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:28.759233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:28.887644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=23 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:38.165303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba403a-cdc0-4d0b-a8b4-73b9aa12e56a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:38.166745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bcba403a-cdc0-4d0b-a8b4-73b9aa12e56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:38.225597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba403a-cdc0-4d0b-a8b4-73b9aa12e56a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:38.229233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:38.229892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:38.230168 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:38.230436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:38.232245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:38.298974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:38.300415 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:38.317183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1285/2566] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:41.617217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:41.619918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:41.705536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:41.707399 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:41.726360 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825fc4b-a668-4212-b39a-3fb5e188dc84 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:41.727661 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64210fbb-b4e8-4029-b2c5-d9a1deab7fe6 service glance] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:41.732014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:41.733568 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:41.733759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:41.733879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:41.734104 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] PATH_INFO: `/v3/users/8c50160e51b54cde9a3f617d71fbf1c0` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:41.734315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:41.734714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:41.735001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:41.735519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c50160e51b54cde9a3f617d71fbf1c0)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:41.736904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:41.737898 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:41.750223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:41.752452 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:41.763360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1282/2567] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:41 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:32:41.803859 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c50160e51b54cde9a3f617d71fbf1c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:41.816280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c50160e51b54cde9a3f617d71fbf1c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:41.825640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c50160e51b54cde9a3f617d71fbf1c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:41.826621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c50160e51b54cde9a3f617d71fbf1c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:41.840894 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c50160e51b54cde9a3f617d71fbf1c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:41.847929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1286/2568] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:41 2022] DELETE /identity/v3/users/8c50160e51b54cde9a3f617d71fbf1c0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:41.887771 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:41.901146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540aa6a-dc0f-4db9-af89-85c58906fa74 service placement] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:41.965640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:41.966548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:41.966826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:41.967621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] PATH_INFO: `/v3/users/724450d4209b43d5ac95d534e78c68a2` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:41.968917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=724450d4209b43d5ac95d534e78c68a2)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:41.970884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.021727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724450d4209b43d5ac95d534e78c68a2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.030701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724450d4209b43d5ac95d534e78c68a2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.031329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724450d4209b43d5ac95d534e78c68a2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.035064 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.037004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:42.051412 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71c00d-f4aa-4749-b05e-369ad76d6d6b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.055446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724450d4209b43d5ac95d534e78c68a2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.056225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.056416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-cb8bab93-bf5a-4f53-a2bf-6af718a504ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724450d4209b43d5ac95d534e78c68a2'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.057018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.057377 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.057883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] PATH_INFO: `/v3/projects/011b296da6654fa38f76eadd7bebae6f` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.059086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011b296da6654fa38f76eadd7bebae6f)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.061292 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.071303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1283/2569] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:41 2022] DELETE /identity/v3/users/724450d4209b43d5ac95d534e78c68a2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:42.075846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018fbe1-908b-49d9-a26c-fca44f04ec6c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.076197 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018fbe1-908b-49d9-a26c-fca44f04ec6c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.077333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018fbe1-908b-49d9-a26c-fca44f04ec6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.080271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:42.099493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7018fbe1-908b-49d9-a26c-fca44f04ec6c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:42.101015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7018fbe1-908b-49d9-a26c-fca44f04ec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:42.106592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:42.121975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011b296da6654fa38f76eadd7bebae6f', 'request_id': 'req-ccd025d3-7949-4520-9b79-45f1de486a75'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.129151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011b296da6654fa38f76eadd7bebae6f', 'request_id': 'req-ccd025d3-7949-4520-9b79-45f1de486a75'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.137784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011b296da6654fa38f76eadd7bebae6f', 'request_id': 'req-ccd025d3-7949-4520-9b79-45f1de486a75'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.149612 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] The token cache is being invalidate because project 011b296da6654fa38f76eadd7bebae6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:42.149931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.150592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-ccd025d3-7949-4520-9b79-45f1de486a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.152121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1287/2570] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:42 2022] DELETE /identity/v3/projects/011b296da6654fa38f76eadd7bebae6f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:42.157990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b3e2-adb9-49d1-8727-b8fd9017fb1d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.158276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b3e2-adb9-49d1-8727-b8fd9017fb1d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.158562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b3e2-adb9-49d1-8727-b8fd9017fb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.170571 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1284/2571] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:32:42.178195 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7a05b3e2-adb9-49d1-8727-b8fd9017fb1d None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:42.179438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7a05b3e2-adb9-49d1-8727-b8fd9017fb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:42.180442 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018fbe1-908b-49d9-a26c-fca44f04ec6c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.181903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7018fbe1-908b-49d9-a26c-fca44f04ec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:42.194265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1288/2572] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:42.196005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c5538-1fa8-426a-8e05-758c3ed28839 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.196285 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c5538-1fa8-426a-8e05-758c3ed28839 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.196558 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c5538-1fa8-426a-8e05-758c3ed28839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.215937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-752c5538-1fa8-426a-8e05-758c3ed28839 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:42.218084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-752c5538-1fa8-426a-8e05-758c3ed28839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:42.226520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1289/2573] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:32:42.230704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c5538-1fa8-426a-8e05-758c3ed28839 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.240172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c5538-1fa8-426a-8e05-758c3ed28839 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.243569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018fbe1-908b-49d9-a26c-fca44f04ec6c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.244728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.245354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.245671 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.245949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.246631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:42.246931 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.247326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c9080-5916-4739-9075-c1b747455654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.247944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9080-5916-4739-9075-c1b747455654 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.248260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9080-5916-4739-9075-c1b747455654 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.248535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9080-5916-4739-9075-c1b747455654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.248689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.258081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c9080-5916-4739-9075-c1b747455654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.259525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c9080-5916-4739-9075-c1b747455654 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.260328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1290/2574] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:32:42.265023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.270225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1285/2575] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:32:42.278306 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55596581-f894-4e23-9d1a-1d4f0a10760d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.283622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b7e30-21ce-4f20-80e5-99beb8a624da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.284233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b7e30-21ce-4f20-80e5-99beb8a624da admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.284514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b7e30-21ce-4f20-80e5-99beb8a624da admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.284788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b7e30-21ce-4f20-80e5-99beb8a624da admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.285477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b7e30-21ce-4f20-80e5-99beb8a624da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.287393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b7e30-21ce-4f20-80e5-99beb8a624da admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.319692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1291/2576] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:42 2022] POST /identity/v3/projects => generated 394 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:42.321311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c9080-5916-4739-9075-c1b747455654 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.332382 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c9080-5916-4739-9075-c1b747455654 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.394248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80602f1d-465d-4eeb-9b93-ac638a26a737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.394876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80602f1d-465d-4eeb-9b93-ac638a26a737 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.395157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80602f1d-465d-4eeb-9b93-ac638a26a737 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.395431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80602f1d-465d-4eeb-9b93-ac638a26a737 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.396101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80602f1d-465d-4eeb-9b93-ac638a26a737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.398115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80602f1d-465d-4eeb-9b93-ac638a26a737 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.424049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1286/2577] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:42 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:42.426317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b7e30-21ce-4f20-80e5-99beb8a624da admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.437147 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b7e30-21ce-4f20-80e5-99beb8a624da admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.441904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d849f-3035-4e55-ae5a-261243ec33d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.442519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d849f-3035-4e55-ae5a-261243ec33d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.442802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d849f-3035-4e55-ae5a-261243ec33d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.443078 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d849f-3035-4e55-ae5a-261243ec33d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.443781 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d849f-3035-4e55-ae5a-261243ec33d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.445430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d849f-3035-4e55-ae5a-261243ec33d3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.456190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1292/2578] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:42 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:42.460095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80602f1d-465d-4eeb-9b93-ac638a26a737 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.471290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80602f1d-465d-4eeb-9b93-ac638a26a737 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.476147 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69ea4f-d3ac-46bf-8ffe-5ccd8e15a890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.476764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ea4f-d3ac-46bf-8ffe-5ccd8e15a890 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.477047 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ea4f-d3ac-46bf-8ffe-5ccd8e15a890 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.477325 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ea4f-d3ac-46bf-8ffe-5ccd8e15a890 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.478058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69ea4f-d3ac-46bf-8ffe-5ccd8e15a890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.479675 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69ea4f-d3ac-46bf-8ffe-5ccd8e15a890 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.490759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1287/2579] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:42 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:42.494656 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d849f-3035-4e55-ae5a-261243ec33d3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.504862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d849f-3035-4e55-ae5a-261243ec33d3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.509583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.510254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.510555 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.510816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] PATH_INFO: `/v3/projects/b49846a224c346c7b9c51798e648a070/users/8ffae7b6e17f42b59c9df8bd49c5941e/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.525789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b49846a224c346c7b9c51798e648a070, user_id=8ffae7b6e17f42b59c9df8bd49c5941e, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.528487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.543844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1293/2580] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:42 2022] PUT /identity/v3/projects/b49846a224c346c7b9c51798e648a070/users/8ffae7b6e17f42b59c9df8bd49c5941e/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:42.554331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.554969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bed7f-1829-4110-9124-1926e0927a67 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.555178 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bed7f-1829-4110-9124-1926e0927a67 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.555622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bed7f-1829-4110-9124-1926e0927a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.557009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:42.586849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-b94bed7f-1829-4110-9124-1926e0927a67 None None] MFA Rules not processed for user `8ffae7b6e17f42b59c9df8bd49c5941e`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:42.588177 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b94bed7f-1829-4110-9124-1926e0927a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:42.608918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:42.630580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4493a8-bb6b-4f1e-b37f-4f5e82275dee admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.635486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.636119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.636398 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.636668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] PATH_INFO: `/v3/projects/6a0a0b446c6b42c8927636a6357b3456` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.637858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a0a0b446c6b42c8927636a6357b3456)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.639841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.658786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1288/2581] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:42.663168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bed7f-1829-4110-9124-1926e0927a67 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.666801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:42.674378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bed7f-1829-4110-9124-1926e0927a67 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.679393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d424bb-65a6-4cd5-a706-89586ae02a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.680010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d424bb-65a6-4cd5-a706-89586ae02a5f admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.680304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d424bb-65a6-4cd5-a706-89586ae02a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.680569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d424bb-65a6-4cd5-a706-89586ae02a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.681229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d424bb-65a6-4cd5-a706-89586ae02a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.683176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d424bb-65a6-4cd5-a706-89586ae02a5f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.691905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a0a0b446c6b42c8927636a6357b3456', 'request_id': 'req-fff199ec-5fe3-45fc-973f-f866aa50a0a1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.698152 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a0a0b446c6b42c8927636a6357b3456', 'request_id': 'req-fff199ec-5fe3-45fc-973f-f866aa50a0a1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.706520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a0a0b446c6b42c8927636a6357b3456', 'request_id': 'req-fff199ec-5fe3-45fc-973f-f866aa50a0a1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.714168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1289/2582] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:42 2022] POST /identity/v3/projects => generated 394 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:42.715175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d424bb-65a6-4cd5-a706-89586ae02a5f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.719608 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] The token cache is being invalidate because project 6a0a0b446c6b42c8927636a6357b3456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:42.719905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.720594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:42.722165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1294/2583] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:42 2022] DELETE /identity/v3/projects/6a0a0b446c6b42c8927636a6357b3456 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:42.783951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d424bb-65a6-4cd5-a706-89586ae02a5f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.788979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd34479-cacb-4a35-b4b2-87a5bc80da4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.789838 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34479-cacb-4a35-b4b2-87a5bc80da4d admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.790119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34479-cacb-4a35-b4b2-87a5bc80da4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.790551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34479-cacb-4a35-b4b2-87a5bc80da4d admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.791243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd34479-cacb-4a35-b4b2-87a5bc80da4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.793245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd34479-cacb-4a35-b4b2-87a5bc80da4d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.819268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1290/2584] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:42 2022] POST /identity/v3/users => generated 304 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:42.820592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.822098 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:42.892167 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff199ec-5fe3-45fc-973f-f866aa50a0a1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.897108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69951c69-d444-4eff-8e8b-9aff2ff58035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.897912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69951c69-d444-4eff-8e8b-9aff2ff58035 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.898231 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69951c69-d444-4eff-8e8b-9aff2ff58035 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.898506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69951c69-d444-4eff-8e8b-9aff2ff58035 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.899211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69951c69-d444-4eff-8e8b-9aff2ff58035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.900839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69951c69-d444-4eff-8e8b-9aff2ff58035 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.912822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1295/2585] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:42 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:42.917338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd34479-cacb-4a35-b4b2-87a5bc80da4d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.928403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd34479-cacb-4a35-b4b2-87a5bc80da4d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:42.933256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709943a4-49cc-4cda-994c-6ceb24da26d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:42.933988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709943a4-49cc-4cda-994c-6ceb24da26d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:42.934272 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709943a4-49cc-4cda-994c-6ceb24da26d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:42.934548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709943a4-49cc-4cda-994c-6ceb24da26d5 admin admin] PATH_INFO: `/v3/projects/087eb911dde04257a012fb6982696eb3/users/463532a420594f75b219a6bf2577a227/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:42.950952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709943a4-49cc-4cda-994c-6ceb24da26d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087eb911dde04257a012fb6982696eb3, user_id=463532a420594f75b219a6bf2577a227, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:42.954978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709943a4-49cc-4cda-994c-6ceb24da26d5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:42.970124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1291/2586] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:42 2022] PUT /identity/v3/projects/087eb911dde04257a012fb6982696eb3/users/463532a420594f75b219a6bf2577a227/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:42.974244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69951c69-d444-4eff-8e8b-9aff2ff58035 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:42.985065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69951c69-d444-4eff-8e8b-9aff2ff58035 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:43.047805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d4d88-8b6f-42c4-b912-a0012b5c3827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:43.048422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4d88-8b6f-42c4-b912-a0012b5c3827 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:43.048698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4d88-8b6f-42c4-b912-a0012b5c3827 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:43.048965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4d88-8b6f-42c4-b912-a0012b5c3827 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:43.049701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d4d88-8b6f-42c4-b912-a0012b5c3827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:43.051286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d4d88-8b6f-42c4-b912-a0012b5c3827 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:43.061006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1296/2587] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:42 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:43.065333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709943a4-49cc-4cda-994c-6ceb24da26d5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:43.075753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709943a4-49cc-4cda-994c-6ceb24da26d5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:43.081088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:43.081797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:43.082082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:43.082395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] PATH_INFO: `/v3/projects/087eb911dde04257a012fb6982696eb3/users/463532a420594f75b219a6bf2577a227/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:43.084377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087eb911dde04257a012fb6982696eb3, user_id=463532a420594f75b219a6bf2577a227, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:43.087116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:43.098324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1292/2588] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:43 2022] PUT /identity/v3/projects/087eb911dde04257a012fb6982696eb3/users/463532a420594f75b219a6bf2577a227/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:43.109974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9bde-2363-4c59-ae5f-0ea664c573cf None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:43.110266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9bde-2363-4c59-ae5f-0ea664c573cf None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:43.110543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9bde-2363-4c59-ae5f-0ea664c573cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:43.138212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f9ba9bde-2363-4c59-ae5f-0ea664c573cf None None] MFA Rules not processed for user `463532a420594f75b219a6bf2577a227`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:43.140023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f9ba9bde-2363-4c59-ae5f-0ea664c573cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:43.170745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:43.209849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1297/2589] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:43 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:43.220696 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:43.222435 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:43.281135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641569f-f70b-454b-9ad1-4d2ff9c11827 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:43.284895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dd513-5aef-4795-b659-cbdfc4bfa90a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:43.285607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dd513-5aef-4795-b659-cbdfc4bfa90a service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:43.285910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dd513-5aef-4795-b659-cbdfc4bfa90a service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:43.286216 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dd513-5aef-4795-b659-cbdfc4bfa90a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:43.292639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:43.297483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dd513-5aef-4795-b659-cbdfc4bfa90a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:43.299505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dd513-5aef-4795-b659-cbdfc4bfa90a service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:43.308917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1293/2590] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:43 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:43.355776 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9bde-2363-4c59-ae5f-0ea664c573cf None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:43.357178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f9ba9bde-2363-4c59-ae5f-0ea664c573cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:43.369455 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9bde-2363-4c59-ae5f-0ea664c573cf None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:43.374136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ac0b2-651a-47d3-ae1a-815bb177904d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:43.374384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac0b2-651a-47d3-ae1a-815bb177904d service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:43.374667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac0b2-651a-47d3-ae1a-815bb177904d service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:43.374967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac0b2-651a-47d3-ae1a-815bb177904d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:43.376836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-985ac0b2-651a-47d3-ae1a-815bb177904d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:43.437373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ac0b2-651a-47d3-ae1a-815bb177904d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:43.438773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ac0b2-651a-47d3-ae1a-815bb177904d service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:43.447670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1298/2591] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:43 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:46.703861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dd513-5aef-4795-b659-cbdfc4bfa90a service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:46.765896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dd513-5aef-4795-b659-cbdfc4bfa90a service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:46.770420 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:46.771055 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:46.771337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:46.771610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] PATH_INFO: `/v3/users/8ffae7b6e17f42b59c9df8bd49c5941e` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:46.772776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ffae7b6e17f42b59c9df8bd49c5941e)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:46.774735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:46.820158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ffae7b6e17f42b59c9df8bd49c5941e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:46.827360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ffae7b6e17f42b59c9df8bd49c5941e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:46.828097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ffae7b6e17f42b59c9df8bd49c5941e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:46.839932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ffae7b6e17f42b59c9df8bd49c5941e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:46.840568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ffae7b6e17f42b59c9df8bd49c5941e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:46.853766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1294/2592] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:46 2022] DELETE /identity/v3/users/8ffae7b6e17f42b59c9df8bd49c5941e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:46.858430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5451-9216-4123-ab90-251c4b8cb375 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:46.858934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5451-9216-4123-ab90-251c4b8cb375 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:46.859134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5451-9216-4123-ab90-251c4b8cb375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:46.879430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-679c5451-9216-4123-ab90-251c4b8cb375 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:46.880636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-679c5451-9216-4123-ab90-251c4b8cb375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:46.939704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1299/2593] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:46.949463 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:46.950853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:47.008450 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0fa79-981d-4a7b-942f-590915ce0728 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:47.012116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a30f17-e269-4832-a96e-69f89989ca75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:47.012737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a30f17-e269-4832-a96e-69f89989ca75 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:47.013019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a30f17-e269-4832-a96e-69f89989ca75 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:47.013291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a30f17-e269-4832-a96e-69f89989ca75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:47.022940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a30f17-e269-4832-a96e-69f89989ca75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:47.024322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a30f17-e269-4832-a96e-69f89989ca75 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:47.033814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1295/2594] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:32:47.314724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c5451-9216-4123-ab90-251c4b8cb375 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:47.316181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-679c5451-9216-4123-ab90-251c4b8cb375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:47.330544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c5451-9216-4123-ab90-251c4b8cb375 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:47.335133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:47.336304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:47.336768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:47.337057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] PATH_INFO: `/v3/projects/b49846a224c346c7b9c51798e648a070` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:47.338249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b49846a224c346c7b9c51798e648a070)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:47.340154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:47.386668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b49846a224c346c7b9c51798e648a070', 'request_id': 'req-5bc57be6-f035-491e-9f56-3089b5821035'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.392976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b49846a224c346c7b9c51798e648a070', 'request_id': 'req-5bc57be6-f035-491e-9f56-3089b5821035'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.396549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:47.400493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b49846a224c346c7b9c51798e648a070', 'request_id': 'req-5bc57be6-f035-491e-9f56-3089b5821035'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.414314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] The token cache is being invalidate because project b49846a224c346c7b9c51798e648a070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:47.414601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.415261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.416770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1300/2595] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:47 2022] DELETE /identity/v3/projects/b49846a224c346c7b9c51798e648a070 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:47.420782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a30f17-e269-4832-a96e-69f89989ca75 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:47.430316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a30f17-e269-4832-a96e-69f89989ca75 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:47.441323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:47.486761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:47.487390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:47.487666 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:47.487934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] PATH_INFO: `/v3/users/463532a420594f75b219a6bf2577a227` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:47.489077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=463532a420594f75b219a6bf2577a227)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:47.491049 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:47.537921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '463532a420594f75b219a6bf2577a227'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.545097 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '463532a420594f75b219a6bf2577a227'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.545753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '463532a420594f75b219a6bf2577a227'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.557843 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '463532a420594f75b219a6bf2577a227'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.558418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ee2e5e7a-08be-4234-8203-8eaf11b3ecb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '463532a420594f75b219a6bf2577a227'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.571763 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1296/2596] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:47 2022] DELETE /identity/v3/users/463532a420594f75b219a6bf2577a227 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:47.839783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:47.841603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:47.855130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc57be6-f035-491e-9f56-3089b5821035 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:47.911876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:47.912590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:47.912792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:47.913067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] PATH_INFO: `/v3/projects/087eb911dde04257a012fb6982696eb3` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:47.914368 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087eb911dde04257a012fb6982696eb3)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:47.916315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:47.961786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087eb911dde04257a012fb6982696eb3', 'request_id': 'req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.967779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087eb911dde04257a012fb6982696eb3', 'request_id': 'req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.975036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087eb911dde04257a012fb6982696eb3', 'request_id': 'req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.986953 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] The token cache is being invalidate because project 087eb911dde04257a012fb6982696eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:47.987239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.987875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-57cf1ba4-ce6b-4807-be16-bd9ca1ea85e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:47.989375 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1301/2597] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:47 2022] DELETE /identity/v3/projects/087eb911dde04257a012fb6982696eb3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:47.994243 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260434ab-d3ad-42c4-a384-921a5e5df789 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:47.994737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260434ab-d3ad-42c4-a384-921a5e5df789 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:47.994840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260434ab-d3ad-42c4-a384-921a5e5df789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.014998 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-260434ab-d3ad-42c4-a384-921a5e5df789 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:48.016230 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-260434ab-d3ad-42c4-a384-921a5e5df789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:48.075108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1297/2598] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:48.082977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd584a94-ae01-4566-9dfa-7018a1524a8c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.083197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd584a94-ae01-4566-9dfa-7018a1524a8c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.083470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd584a94-ae01-4566-9dfa-7018a1524a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.103108 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-bd584a94-ae01-4566-9dfa-7018a1524a8c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:48.104305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bd584a94-ae01-4566-9dfa-7018a1524a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:48.113015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1302/2599] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:48.117303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260434ab-d3ad-42c4-a384-921a5e5df789 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.127901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260434ab-d3ad-42c4-a384-921a5e5df789 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.132531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.133145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.133460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.133738 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.134438 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:48.134740 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.136431 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.148009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1298/2600] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:32:48.152096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd584a94-ae01-4566-9dfa-7018a1524a8c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.161530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd584a94-ae01-4566-9dfa-7018a1524a8c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.166197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aeb28-8dbc-4346-b297-b15003982bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.166800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aeb28-8dbc-4346-b297-b15003982bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.167083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aeb28-8dbc-4346-b297-b15003982bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.167358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aeb28-8dbc-4346-b297-b15003982bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.168025 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96aeb28-8dbc-4346-b297-b15003982bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.170549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96aeb28-8dbc-4346-b297-b15003982bc0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.199644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1303/2601] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:48 2022] POST /identity/v3/projects => generated 382 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:48.203799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.213979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506538-6349-4bc4-982c-79125ecd75bf admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.232926 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:48.272125 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2c358-da80-4e5b-9d50-d17cccdcc0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.272746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2c358-da80-4e5b-9d50-d17cccdcc0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.273028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2c358-da80-4e5b-9d50-d17cccdcc0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.273304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2c358-da80-4e5b-9d50-d17cccdcc0e5 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.273999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2c358-da80-4e5b-9d50-d17cccdcc0e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.275921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2c358-da80-4e5b-9d50-d17cccdcc0e5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.295942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1299/2602] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:48 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:48.299963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aeb28-8dbc-4346-b297-b15003982bc0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.310086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aeb28-8dbc-4346-b297-b15003982bc0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.315289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a171e4-02e1-449b-8edb-249df1acbef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.315896 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a171e4-02e1-449b-8edb-249df1acbef5 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.316175 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a171e4-02e1-449b-8edb-249df1acbef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.316450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a171e4-02e1-449b-8edb-249df1acbef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.317146 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a171e4-02e1-449b-8edb-249df1acbef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.318774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a171e4-02e1-449b-8edb-249df1acbef5 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.328797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1304/2603] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:48 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:32:48.332521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2c358-da80-4e5b-9d50-d17cccdcc0e5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.342827 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2c358-da80-4e5b-9d50-d17cccdcc0e5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.347330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c85bcb-ccd9-47e6-84fd-c086d961ad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.347930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c85bcb-ccd9-47e6-84fd-c086d961ad74 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.348210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c85bcb-ccd9-47e6-84fd-c086d961ad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.348480 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c85bcb-ccd9-47e6-84fd-c086d961ad74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.349165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c85bcb-ccd9-47e6-84fd-c086d961ad74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.350804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c85bcb-ccd9-47e6-84fd-c086d961ad74 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.361788 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1300/2604] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:48 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:48.365696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a171e4-02e1-449b-8edb-249df1acbef5 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.376048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a171e4-02e1-449b-8edb-249df1acbef5 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.380582 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794fb5e-ffff-4c6b-9f3b-2b71148586fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.381223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794fb5e-ffff-4c6b-9f3b-2b71148586fc admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.381530 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794fb5e-ffff-4c6b-9f3b-2b71148586fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.381801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794fb5e-ffff-4c6b-9f3b-2b71148586fc admin admin] PATH_INFO: `/v3/projects/f58d4d17e47a42e48eafa6aa35618579/users/2df855e8ce754f309ebc04a69f8c9984/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.395860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794fb5e-ffff-4c6b-9f3b-2b71148586fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58d4d17e47a42e48eafa6aa35618579, user_id=2df855e8ce754f309ebc04a69f8c9984, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.398562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794fb5e-ffff-4c6b-9f3b-2b71148586fc admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.411154 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1305/2605] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:48 2022] PUT /identity/v3/projects/f58d4d17e47a42e48eafa6aa35618579/users/2df855e8ce754f309ebc04a69f8c9984/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:48.423408 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee42ca-8990-4712-a1f0-b449277e61ed None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.423694 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee42ca-8990-4712-a1f0-b449277e61ed None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.423967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee42ca-8990-4712-a1f0-b449277e61ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.450889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-35ee42ca-8990-4712-a1f0-b449277e61ed None None] MFA Rules not processed for user `2df855e8ce754f309ebc04a69f8c9984`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:48.452113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-35ee42ca-8990-4712-a1f0-b449277e61ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:48.515995 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1301/2606] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:48.520647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794fb5e-ffff-4c6b-9f3b-2b71148586fc admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.539695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:48.584036 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794fb5e-ffff-4c6b-9f3b-2b71148586fc admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.588886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db8079-e39a-45f0-bcd5-8013ba533995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.589541 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db8079-e39a-45f0-bcd5-8013ba533995 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.589827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db8079-e39a-45f0-bcd5-8013ba533995 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.590100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db8079-e39a-45f0-bcd5-8013ba533995 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.590784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db8079-e39a-45f0-bcd5-8013ba533995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.593065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db8079-e39a-45f0-bcd5-8013ba533995 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.618897 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1306/2607] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:48 2022] POST /identity/v3/projects => generated 382 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:48.623551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee42ca-8990-4712-a1f0-b449277e61ed None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.634222 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee42ca-8990-4712-a1f0-b449277e61ed None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.692029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc332e-1853-462c-8b49-bcf124e55ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.692638 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc332e-1853-462c-8b49-bcf124e55ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.692924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc332e-1853-462c-8b49-bcf124e55ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.693203 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc332e-1853-462c-8b49-bcf124e55ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.693915 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc332e-1853-462c-8b49-bcf124e55ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.695845 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc332e-1853-462c-8b49-bcf124e55ba5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.717513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1302/2608] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:48 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:48.721615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db8079-e39a-45f0-bcd5-8013ba533995 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.731814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db8079-e39a-45f0-bcd5-8013ba533995 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.737255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24758beb-35ba-4697-a2d9-70ebcebc8b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.737926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24758beb-35ba-4697-a2d9-70ebcebc8b46 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.738207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24758beb-35ba-4697-a2d9-70ebcebc8b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.738476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24758beb-35ba-4697-a2d9-70ebcebc8b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.739171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24758beb-35ba-4697-a2d9-70ebcebc8b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.740760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24758beb-35ba-4697-a2d9-70ebcebc8b46 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.751390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1307/2609] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:48 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:48.755407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc332e-1853-462c-8b49-bcf124e55ba5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.765791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc332e-1853-462c-8b49-bcf124e55ba5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.770356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c3ae9-829d-4faf-badb-f584ac09e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.771004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3ae9-829d-4faf-badb-f584ac09e1ba admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.771273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3ae9-829d-4faf-badb-f584ac09e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.771550 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3ae9-829d-4faf-badb-f584ac09e1ba admin admin] PATH_INFO: `/v3/projects/9be81d2ed5ed4b7bbc6dee68220cc922/users/d13ca69a4fbf483b8fc89b350c09ef6e/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.786509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c3ae9-829d-4faf-badb-f584ac09e1ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be81d2ed5ed4b7bbc6dee68220cc922, user_id=d13ca69a4fbf483b8fc89b350c09ef6e, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.789159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c3ae9-829d-4faf-badb-f584ac09e1ba admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.792136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:48.803322 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1303/2610] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:48 2022] PUT /identity/v3/projects/9be81d2ed5ed4b7bbc6dee68220cc922/users/d13ca69a4fbf483b8fc89b350c09ef6e/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:48.807284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24758beb-35ba-4697-a2d9-70ebcebc8b46 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.817432 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24758beb-35ba-4697-a2d9-70ebcebc8b46 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.874436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f35857-0584-4b3e-b73e-494c1fc2104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.874942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f35857-0584-4b3e-b73e-494c1fc2104f admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.875219 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f35857-0584-4b3e-b73e-494c1fc2104f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.875487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f35857-0584-4b3e-b73e-494c1fc2104f admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.876225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f35857-0584-4b3e-b73e-494c1fc2104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.877841 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f35857-0584-4b3e-b73e-494c1fc2104f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.887559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1308/2611] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:48 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:48.892505 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c3ae9-829d-4faf-badb-f584ac09e1ba admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:48.902910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c3ae9-829d-4faf-badb-f584ac09e1ba admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:48.908202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:48.908868 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.909151 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.909454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] PATH_INFO: `/v3/projects/9be81d2ed5ed4b7bbc6dee68220cc922/users/d13ca69a4fbf483b8fc89b350c09ef6e/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.911419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be81d2ed5ed4b7bbc6dee68220cc922, user_id=d13ca69a4fbf483b8fc89b350c09ef6e, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:48.914120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:48.924361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1304/2612] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:48 2022] PUT /identity/v3/projects/9be81d2ed5ed4b7bbc6dee68220cc922/users/d13ca69a4fbf483b8fc89b350c09ef6e/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:48.934308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78116-dc45-4cfe-a6b4-b816e9cb1088 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:48.934802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78116-dc45-4cfe-a6b4-b816e9cb1088 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:48.934904 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78116-dc45-4cfe-a6b4-b816e9cb1088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:48.964031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e8d78116-dc45-4cfe-a6b4-b816e9cb1088 None None] MFA Rules not processed for user `d13ca69a4fbf483b8fc89b350c09ef6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:48.964636 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e8d78116-dc45-4cfe-a6b4-b816e9cb1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:49.034813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1309/2613] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:48 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:49.046223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:49.048006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:49.109156 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaecca-6095-43b4-a9bc-4cf2f9eb0e6e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:49.112872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac882f-ac37-4a87-85f2-a906cad0e3ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:49.113553 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac882f-ac37-4a87-85f2-a906cad0e3ce service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:49.113858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac882f-ac37-4a87-85f2-a906cad0e3ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:49.114129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac882f-ac37-4a87-85f2-a906cad0e3ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:49.124397 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac882f-ac37-4a87-85f2-a906cad0e3ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:49.125930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac882f-ac37-4a87-85f2-a906cad0e3ce service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:49.135307 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1305/2614] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:49 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:49.238278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78116-dc45-4cfe-a6b4-b816e9cb1088 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:49.247889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78116-dc45-4cfe-a6b4-b816e9cb1088 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:49.251472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeab2c6-58b5-40f8-b6e0-5ed4cb80e0f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:49.252072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeab2c6-58b5-40f8-b6e0-5ed4cb80e0f2 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:49.252345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeab2c6-58b5-40f8-b6e0-5ed4cb80e0f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:49.252613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeab2c6-58b5-40f8-b6e0-5ed4cb80e0f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:49.261623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:49.309945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeab2c6-58b5-40f8-b6e0-5ed4cb80e0f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:49.311446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeab2c6-58b5-40f8-b6e0-5ed4cb80e0f2 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:49.320421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1310/2615] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:49 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:51.893090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac882f-ac37-4a87-85f2-a906cad0e3ce service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:51.954797 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac882f-ac37-4a87-85f2-a906cad0e3ce service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:51.960013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:51.960662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:51.960946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:51.961227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] PATH_INFO: `/v3/users/2df855e8ce754f309ebc04a69f8c9984` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:51.962448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df855e8ce754f309ebc04a69f8c9984)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:51.964402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:52.011385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df855e8ce754f309ebc04a69f8c9984'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.018752 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df855e8ce754f309ebc04a69f8c9984'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.019368 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df855e8ce754f309ebc04a69f8c9984'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.031765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df855e8ce754f309ebc04a69f8c9984'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.032389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df855e8ce754f309ebc04a69f8c9984'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.045173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1306/2616] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:51 2022] DELETE /identity/v3/users/2df855e8ce754f309ebc04a69f8c9984 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:52.049601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9976be-6903-4cee-acb3-da328fec00a6 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:52.050090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9976be-6903-4cee-acb3-da328fec00a6 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:52.050207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9976be-6903-4cee-acb3-da328fec00a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:52.068925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9f9976be-6903-4cee-acb3-da328fec00a6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:52.070157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9f9976be-6903-4cee-acb3-da328fec00a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:52.130992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1311/2617] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:52.141400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:52.142867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:52.149219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:52.202841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641678-9c63-4afc-80b1-e96caa071a45 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:52.206716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0460-b03e-4312-9975-a8c057826648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:52.207229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0460-b03e-4312-9975-a8c057826648 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:52.207599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0460-b03e-4312-9975-a8c057826648 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:52.207886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0460-b03e-4312-9975-a8c057826648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:52.217597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d0460-b03e-4312-9975-a8c057826648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:52.219085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d0460-b03e-4312-9975-a8c057826648 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:52.228449 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1307/2618] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:32:52.498514 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9976be-6903-4cee-acb3-da328fec00a6 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:52.500054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9f9976be-6903-4cee-acb3-da328fec00a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:52.512515 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9976be-6903-4cee-acb3-da328fec00a6 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:52.517594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:52.518511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:52.518796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:52.519068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] PATH_INFO: `/v3/projects/f58d4d17e47a42e48eafa6aa35618579` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:52.520574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58d4d17e47a42e48eafa6aa35618579)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:52.522537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:52.573831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58d4d17e47a42e48eafa6aa35618579', 'request_id': 'req-4e641864-4ab6-4cf9-ba23-3494eee34e46'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.581229 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58d4d17e47a42e48eafa6aa35618579', 'request_id': 'req-4e641864-4ab6-4cf9-ba23-3494eee34e46'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.588823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58d4d17e47a42e48eafa6aa35618579', 'request_id': 'req-4e641864-4ab6-4cf9-ba23-3494eee34e46'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.600307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] The token cache is being invalidate because project f58d4d17e47a42e48eafa6aa35618579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:52.600594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.601220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.602759 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1312/2619] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:52 2022] DELETE /identity/v3/projects/f58d4d17e47a42e48eafa6aa35618579 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:52.606663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0460-b03e-4312-9975-a8c057826648 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:52.616930 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0460-b03e-4312-9975-a8c057826648 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:52.678952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:52.680016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:52.680306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:52.680586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] PATH_INFO: `/v3/users/d13ca69a4fbf483b8fc89b350c09ef6e` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:52.681809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13ca69a4fbf483b8fc89b350c09ef6e)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:52.683795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:52.735332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13ca69a4fbf483b8fc89b350c09ef6e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.743457 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13ca69a4fbf483b8fc89b350c09ef6e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.743983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13ca69a4fbf483b8fc89b350c09ef6e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.756756 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13ca69a4fbf483b8fc89b350c09ef6e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.757429 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ed0375b9-38da-42e4-8369-472143bfb8e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13ca69a4fbf483b8fc89b350c09ef6e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:52.770038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1308/2620] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:32:52 2022] DELETE /identity/v3/users/d13ca69a4fbf483b8fc89b350c09ef6e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:53.084629 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:53.086267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:53.106576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e641864-4ab6-4cf9-ba23-3494eee34e46 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:53.151027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:53.181602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:53.182628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.183093 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.183543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] PATH_INFO: `/v3/projects/9be81d2ed5ed4b7bbc6dee68220cc922` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.185397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be81d2ed5ed4b7bbc6dee68220cc922)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:53.188574 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:53.244598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be81d2ed5ed4b7bbc6dee68220cc922', 'request_id': 'req-f066b61b-fe56-49dc-9d2a-98aea9b89f43'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:53.251090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be81d2ed5ed4b7bbc6dee68220cc922', 'request_id': 'req-f066b61b-fe56-49dc-9d2a-98aea9b89f43'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:53.258643 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be81d2ed5ed4b7bbc6dee68220cc922', 'request_id': 'req-f066b61b-fe56-49dc-9d2a-98aea9b89f43'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:53.270750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] The token cache is being invalidate because project 9be81d2ed5ed4b7bbc6dee68220cc922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:32:53.271031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:53.271723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f066b61b-fe56-49dc-9d2a-98aea9b89f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:32:53.273304 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1313/2621] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:32:53 2022] DELETE /identity/v3/projects/9be81d2ed5ed4b7bbc6dee68220cc922 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:53.279996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798b99b-3887-4ccc-be5b-9602ba3bdfc4 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.280523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798b99b-3887-4ccc-be5b-9602ba3bdfc4 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.280856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798b99b-3887-4ccc-be5b-9602ba3bdfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.305466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-1798b99b-3887-4ccc-be5b-9602ba3bdfc4 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:53.306712 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-1798b99b-3887-4ccc-be5b-9602ba3bdfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:53.329573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:53.373778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1309/2622] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:53.382276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905c0a-6a43-47e1-9963-abd368aee376 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.382559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905c0a-6a43-47e1-9963-abd368aee376 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.382834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905c0a-6a43-47e1-9963-abd368aee376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.537224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7a905c0a-6a43-47e1-9963-abd368aee376 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:53.538396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7a905c0a-6a43-47e1-9963-abd368aee376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:53.546554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1314/2623] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:53.550979 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798b99b-3887-4ccc-be5b-9602ba3bdfc4 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:53.560749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798b99b-3887-4ccc-be5b-9602ba3bdfc4 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:53.565289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:53.565929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.566205 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.566474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.567157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:32:53.567454 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:53.569048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:53.580945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1310/2624] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:32:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:32:53.585319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a905c0a-6a43-47e1-9963-abd368aee376 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:53.595542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a905c0a-6a43-47e1-9963-abd368aee376 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:53.600198 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31ddb7-ad34-4d80-a2f7-1e46fb66fce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:53.600804 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31ddb7-ad34-4d80-a2f7-1e46fb66fce3 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.601084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31ddb7-ad34-4d80-a2f7-1e46fb66fce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.601353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31ddb7-ad34-4d80-a2f7-1e46fb66fce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.602040 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31ddb7-ad34-4d80-a2f7-1e46fb66fce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:53.603917 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31ddb7-ad34-4d80-a2f7-1e46fb66fce3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:53.630836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1315/2625] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:53 2022] POST /identity/v3/projects => generated 408 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:53.635081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:53.645521 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21617bd-3303-4062-8f16-68555ad999b8 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:53.701128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1f67d-a53f-40eb-9106-88f3985b0942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:53.701781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f67d-a53f-40eb-9106-88f3985b0942 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.702059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f67d-a53f-40eb-9106-88f3985b0942 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.702329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f67d-a53f-40eb-9106-88f3985b0942 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.702981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1f67d-a53f-40eb-9106-88f3985b0942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:53.704891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1f67d-a53f-40eb-9106-88f3985b0942 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:53.725567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1311/2626] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:53 2022] POST /identity/v3/users => generated 305 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:53.729847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31ddb7-ad34-4d80-a2f7-1e46fb66fce3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:53.741376 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31ddb7-ad34-4d80-a2f7-1e46fb66fce3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:53.746020 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde78588-b4c6-4867-9fdf-01d96da95b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:53.746630 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde78588-b4c6-4867-9fdf-01d96da95b56 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.746909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde78588-b4c6-4867-9fdf-01d96da95b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.747179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde78588-b4c6-4867-9fdf-01d96da95b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.748143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde78588-b4c6-4867-9fdf-01d96da95b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:53.749757 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde78588-b4c6-4867-9fdf-01d96da95b56 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:53.759532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1316/2627] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:53 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:53.763631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1f67d-a53f-40eb-9106-88f3985b0942 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:53.773835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1f67d-a53f-40eb-9106-88f3985b0942 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:53.778413 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44eeb4-67a7-4885-8d99-9e2fd0463f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:53.779594 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44eeb4-67a7-4885-8d99-9e2fd0463f79 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.779871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44eeb4-67a7-4885-8d99-9e2fd0463f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.780141 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44eeb4-67a7-4885-8d99-9e2fd0463f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.780828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44eeb4-67a7-4885-8d99-9e2fd0463f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:53.782447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44eeb4-67a7-4885-8d99-9e2fd0463f79 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:53.792954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1312/2628] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:53 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:53.797653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde78588-b4c6-4867-9fdf-01d96da95b56 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:53.807424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde78588-b4c6-4867-9fdf-01d96da95b56 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:53.812047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cc3dc-5535-4313-bd68-f5cdf360b458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:53.812701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cc3dc-5535-4313-bd68-f5cdf360b458 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.812981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cc3dc-5535-4313-bd68-f5cdf360b458 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.813256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cc3dc-5535-4313-bd68-f5cdf360b458 admin admin] PATH_INFO: `/v3/projects/9db0454130984b1d82499efe820bc486/users/36b9a61074e5457dafaf0fe591e8aa6b/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.828140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cc3dc-5535-4313-bd68-f5cdf360b458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db0454130984b1d82499efe820bc486, user_id=36b9a61074e5457dafaf0fe591e8aa6b, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:53.830855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cc3dc-5535-4313-bd68-f5cdf360b458 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:53.845213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1317/2629] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:53 2022] PUT /identity/v3/projects/9db0454130984b1d82499efe820bc486/users/36b9a61074e5457dafaf0fe591e8aa6b/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:53.856650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b673-1d94-480a-a7d4-5d5fb3f6643e None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:53.857085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b673-1d94-480a-a7d4-5d5fb3f6643e None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:53.857357 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b673-1d94-480a-a7d4-5d5fb3f6643e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:53.886026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-5534b673-1d94-480a-a7d4-5d5fb3f6643e None None] MFA Rules not processed for user `36b9a61074e5457dafaf0fe591e8aa6b`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:53.887275 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5534b673-1d94-480a-a7d4-5d5fb3f6643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:53.950130 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1313/2630] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:53 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:32:53.956054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cc3dc-5535-4313-bd68-f5cdf360b458 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:54.021890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cc3dc-5535-4313-bd68-f5cdf360b458 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:54.026446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf2e20-8f09-4c2d-8af2-8158cd287228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:54.027045 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2e20-8f09-4c2d-8af2-8158cd287228 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.027317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2e20-8f09-4c2d-8af2-8158cd287228 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.027581 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2e20-8f09-4c2d-8af2-8158cd287228 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.028226 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf2e20-8f09-4c2d-8af2-8158cd287228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:54.030183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf2e20-8f09-4c2d-8af2-8158cd287228 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:54.035257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:54.058427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1318/2631] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:32:53 2022] POST /identity/v3/projects => generated 406 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:32:54.063187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534b673-1d94-480a-a7d4-5d5fb3f6643e None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:54.074136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534b673-1d94-480a-a7d4-5d5fb3f6643e None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:54.092846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:54.135194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a11ffa-4ffa-43a3-950e-29a65ffb1d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:54.135814 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a11ffa-4ffa-43a3-950e-29a65ffb1d9d admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.136091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a11ffa-4ffa-43a3-950e-29a65ffb1d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.136362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a11ffa-4ffa-43a3-950e-29a65ffb1d9d admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.137026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a11ffa-4ffa-43a3-950e-29a65ffb1d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:54.139601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a11ffa-4ffa-43a3-950e-29a65ffb1d9d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:54.163509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1314/2632] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:32:54 2022] POST /identity/v3/users => generated 310 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:32:54.167127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf2e20-8f09-4c2d-8af2-8158cd287228 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:54.177521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf2e20-8f09-4c2d-8af2-8158cd287228 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:54.182053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6f459-ef8c-44eb-a518-9d6910060229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:54.182653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6f459-ef8c-44eb-a518-9d6910060229 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.182934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6f459-ef8c-44eb-a518-9d6910060229 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.183186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6f459-ef8c-44eb-a518-9d6910060229 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.183862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6f459-ef8c-44eb-a518-9d6910060229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:54.185487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6f459-ef8c-44eb-a518-9d6910060229 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:54.195157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1319/2633] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:54 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:54.199259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a11ffa-4ffa-43a3-950e-29a65ffb1d9d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:54.210172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a11ffa-4ffa-43a3-950e-29a65ffb1d9d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:54.214884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99053171-c704-4f6c-b7a0-6d5716ced01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:54.215567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053171-c704-4f6c-b7a0-6d5716ced01d admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.216742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053171-c704-4f6c-b7a0-6d5716ced01d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.217080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053171-c704-4f6c-b7a0-6d5716ced01d admin admin] PATH_INFO: `/v3/projects/11d1453493ad453b95476ef242d2938d/users/fdf249b3ab16418ab07d71dc2909b31c/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.232447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053171-c704-4f6c-b7a0-6d5716ced01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d1453493ad453b95476ef242d2938d, user_id=fdf249b3ab16418ab07d71dc2909b31c, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:54.235297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053171-c704-4f6c-b7a0-6d5716ced01d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:54.248901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1315/2634] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:54 2022] PUT /identity/v3/projects/11d1453493ad453b95476ef242d2938d/users/fdf249b3ab16418ab07d71dc2909b31c/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:54.252982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6f459-ef8c-44eb-a518-9d6910060229 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:54.263512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6f459-ef8c-44eb-a518-9d6910060229 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:54.320869 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf6581-ad05-419f-96a8-4e0da49f81b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:54.321397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6581-ad05-419f-96a8-4e0da49f81b4 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.321703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6581-ad05-419f-96a8-4e0da49f81b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.321976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6581-ad05-419f-96a8-4e0da49f81b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.322654 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf6581-ad05-419f-96a8-4e0da49f81b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:54.324227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf6581-ad05-419f-96a8-4e0da49f81b4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:54.334915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1320/2635] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:32:54 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:32:54.339124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99053171-c704-4f6c-b7a0-6d5716ced01d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:54.350483 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99053171-c704-4f6c-b7a0-6d5716ced01d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:54.355158 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:54.355819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.356103 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.356381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] PATH_INFO: `/v3/projects/11d1453493ad453b95476ef242d2938d/users/fdf249b3ab16418ab07d71dc2909b31c/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.358746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d1453493ad453b95476ef242d2938d, user_id=fdf249b3ab16418ab07d71dc2909b31c, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:54.361415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:54.371619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1316/2636] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:32:54 2022] PUT /identity/v3/projects/11d1453493ad453b95476ef242d2938d/users/fdf249b3ab16418ab07d71dc2909b31c/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:32:54.382686 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46e64-40a6-40c4-ba26-58cf18de2e74 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.383006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46e64-40a6-40c4-ba26-58cf18de2e74 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.383283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46e64-40a6-40c4-ba26-58cf18de2e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.410963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-65b46e64-40a6-40c4-ba26-58cf18de2e74 None None] MFA Rules not processed for user `fdf249b3ab16418ab07d71dc2909b31c`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:32:54.412209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-65b46e64-40a6-40c4-ba26-58cf18de2e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:54.479116 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1321/2637] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:32:54 2022] POST /identity/v3/auth/tokens => generated 3359 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:32:54.489794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:54.491519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:32:54.549654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2020-8846-4039-a987-af4fd48cad8b admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:54.553213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b546310-4abd-4b44-855a-da9a438979de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:54.553857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b546310-4abd-4b44-855a-da9a438979de service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.554134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b546310-4abd-4b44-855a-da9a438979de service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.554402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b546310-4abd-4b44-855a-da9a438979de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.563689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b546310-4abd-4b44-855a-da9a438979de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:54.565112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b546310-4abd-4b44-855a-da9a438979de service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:54.574593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1317/2638] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:54 2022] GET /identity/v3/auth/tokens => generated 3359 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:32:54.722144 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46e64-40a6-40c4-ba26-58cf18de2e74 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:32:54.732169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46e64-40a6-40c4-ba26-58cf18de2e74 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:32:54.735707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf26500-29b2-4746-9168-b38c6243f324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:32:54.736312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf26500-29b2-4746-9168-b38c6243f324 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:32:54.736617 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf26500-29b2-4746-9168-b38c6243f324 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:32:54.736884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf26500-29b2-4746-9168-b38c6243f324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:32:54.789174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:32:54.791507 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf26500-29b2-4746-9168-b38c6243f324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:32:54.793276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf26500-29b2-4746-9168-b38c6243f324 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:32:54.803173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1322/2639] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:32:54 2022] GET /identity/v3/auth/tokens => generated 3294 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:04.168557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b546310-4abd-4b44-855a-da9a438979de service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:04.230389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b546310-4abd-4b44-855a-da9a438979de service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:04.235491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:04.236131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:04.236412 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:04.236688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] PATH_INFO: `/v3/users/36b9a61074e5457dafaf0fe591e8aa6b` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:04.237879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b9a61074e5457dafaf0fe591e8aa6b)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:04.239833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:04.260699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:04.284902 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b9a61074e5457dafaf0fe591e8aa6b'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.291464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b9a61074e5457dafaf0fe591e8aa6b'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.292206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b9a61074e5457dafaf0fe591e8aa6b'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.305127 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b9a61074e5457dafaf0fe591e8aa6b'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.305809 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b9a61074e5457dafaf0fe591e8aa6b'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.318424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1318/2640] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:33:04 2022] DELETE /identity/v3/users/36b9a61074e5457dafaf0fe591e8aa6b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:04.323079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af92591-eb5d-4e95-8b7a-3ff5b88b87c3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:04.323577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af92591-eb5d-4e95-8b7a-3ff5b88b87c3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:04.323676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af92591-eb5d-4e95-8b7a-3ff5b88b87c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:04.344347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9af92591-eb5d-4e95-8b7a-3ff5b88b87c3 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:04.345585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9af92591-eb5d-4e95-8b7a-3ff5b88b87c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:04.402130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1323/2641] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:04.412876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:04.414291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:04.471813 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7643b2-23bc-4fe1-826a-4d20ae0fc649 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:04.475509 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58258053-abd8-42cc-8dc8-329bc5b92e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:04.476132 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258053-abd8-42cc-8dc8-329bc5b92e28 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:04.476416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258053-abd8-42cc-8dc8-329bc5b92e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:04.476689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58258053-abd8-42cc-8dc8-329bc5b92e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:04.486112 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58258053-abd8-42cc-8dc8-329bc5b92e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:04.487552 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58258053-abd8-42cc-8dc8-329bc5b92e28 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:04.496953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1319/2642] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:04.762069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af92591-eb5d-4e95-8b7a-3ff5b88b87c3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:04.764212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9af92591-eb5d-4e95-8b7a-3ff5b88b87c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:04.776516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af92591-eb5d-4e95-8b7a-3ff5b88b87c3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:04.782702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:04.783509 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:04.783670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:04.783962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] PATH_INFO: `/v3/projects/9db0454130984b1d82499efe820bc486` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:04.785131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db0454130984b1d82499efe820bc486)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:04.787543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:04.841393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db0454130984b1d82499efe820bc486', 'request_id': 'req-d840a1cb-c05e-4f70-a88c-89fe68d25881'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.847353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db0454130984b1d82499efe820bc486', 'request_id': 'req-d840a1cb-c05e-4f70-a88c-89fe68d25881'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.855945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db0454130984b1d82499efe820bc486', 'request_id': 'req-d840a1cb-c05e-4f70-a88c-89fe68d25881'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.868790 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] The token cache is being invalidate because project 9db0454130984b1d82499efe820bc486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:33:04.869082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.869758 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:04.871338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1324/2643] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:33:04 2022] DELETE /identity/v3/projects/9db0454130984b1d82499efe820bc486 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:04.876321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58258053-abd8-42cc-8dc8-329bc5b92e28 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:04.886613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58258053-abd8-42cc-8dc8-329bc5b92e28 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:04.952761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:04.953419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:04.953764 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:04.954075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] PATH_INFO: `/v3/users/fdf249b3ab16418ab07d71dc2909b31c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:04.955620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf249b3ab16418ab07d71dc2909b31c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:04.957819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:04.998965 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:05.005038 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf249b3ab16418ab07d71dc2909b31c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.013715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf249b3ab16418ab07d71dc2909b31c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.014272 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf249b3ab16418ab07d71dc2909b31c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.026108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf249b3ab16418ab07d71dc2909b31c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.026742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-ad1a7e84-5cbe-4105-9063-73c7f6c03d04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf249b3ab16418ab07d71dc2909b31c'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.038593 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1320/2644] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:33:04 2022] DELETE /identity/v3/users/fdf249b3ab16418ab07d71dc2909b31c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:05.318543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:05.319960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:05.333286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840a1cb-c05e-4f70-a88c-89fe68d25881 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:05.411151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:05.411798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.412084 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.412355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] PATH_INFO: `/v3/projects/11d1453493ad453b95476ef242d2938d` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.413548 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d1453493ad453b95476ef242d2938d)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:05.415434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:05.437988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:05.461963 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d1453493ad453b95476ef242d2938d', 'request_id': 'req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.468962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d1453493ad453b95476ef242d2938d', 'request_id': 'req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.475540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d1453493ad453b95476ef242d2938d', 'request_id': 'req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.487386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] The token cache is being invalidate because project 11d1453493ad453b95476ef242d2938d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:33:05.487672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.488319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-c29c1d8b-5ceb-4dd0-9fe2-e91f769dd09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:05.489834 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1325/2645] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:33:05 2022] DELETE /identity/v3/projects/11d1453493ad453b95476ef242d2938d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:05.494656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb22dc-8aea-416c-9787-ef30b0fc1977 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.495152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb22dc-8aea-416c-9787-ef30b0fc1977 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.495255 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb22dc-8aea-416c-9787-ef30b0fc1977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.514819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-7dfb22dc-8aea-416c-9787-ef30b0fc1977 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:05.516016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-7dfb22dc-8aea-416c-9787-ef30b0fc1977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:05.577535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1321/2646] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:05.586180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c869f67-3d0d-4e85-8eb6-359c806c3d45 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.586465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c869f67-3d0d-4e85-8eb6-359c806c3d45 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.586735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c869f67-3d0d-4e85-8eb6-359c806c3d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.608404 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5c869f67-3d0d-4e85-8eb6-359c806c3d45 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:05.609925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5c869f67-3d0d-4e85-8eb6-359c806c3d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:05.618126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1326/2647] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:05.622427 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb22dc-8aea-416c-9787-ef30b0fc1977 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:05.632518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb22dc-8aea-416c-9787-ef30b0fc1977 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:05.637150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:05.637826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.638635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.638904 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.639775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:33:05.640073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:05.641702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:05.657479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1322/2648] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:33:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:33:05.658708 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c869f67-3d0d-4e85-8eb6-359c806c3d45 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:05.668044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c869f67-3d0d-4e85-8eb6-359c806c3d45 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:05.673433 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17145c-b12e-4001-bc14-09b3a807daf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:05.674133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17145c-b12e-4001-bc14-09b3a807daf0 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.674408 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17145c-b12e-4001-bc14-09b3a807daf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.674900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17145c-b12e-4001-bc14-09b3a807daf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.675570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17145c-b12e-4001-bc14-09b3a807daf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:05.677508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17145c-b12e-4001-bc14-09b3a807daf0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:05.706588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1327/2649] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:33:05 2022] POST /identity/v3/projects => generated 372 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:33:05.710592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:05.721799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be244030-8bd7-40fb-a1bb-10d0faa2713d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:05.780201 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b5090-cd52-4d89-8fa4-4bb2493154ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:05.780840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5090-cd52-4d89-8fa4-4bb2493154ad admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.781122 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5090-cd52-4d89-8fa4-4bb2493154ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.781396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5090-cd52-4d89-8fa4-4bb2493154ad admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.782120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b5090-cd52-4d89-8fa4-4bb2493154ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:05.784096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b5090-cd52-4d89-8fa4-4bb2493154ad admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:05.804883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1323/2650] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:33:05 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:33:05.808965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17145c-b12e-4001-bc14-09b3a807daf0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:05.820233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17145c-b12e-4001-bc14-09b3a807daf0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:05.824770 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f470ffa-20cf-4857-b595-897183b80246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:05.825389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470ffa-20cf-4857-b595-897183b80246 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.826197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470ffa-20cf-4857-b595-897183b80246 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.826468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470ffa-20cf-4857-b595-897183b80246 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.827184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f470ffa-20cf-4857-b595-897183b80246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:05.828822 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f470ffa-20cf-4857-b595-897183b80246 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:05.838492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1328/2651] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:05 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:33:05.843501 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b5090-cd52-4d89-8fa4-4bb2493154ad admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:05.853447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b5090-cd52-4d89-8fa4-4bb2493154ad admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:05.858101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ba7c3-9d35-46bc-8dd9-d771e1ea4f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:05.858701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba7c3-9d35-46bc-8dd9-d771e1ea4f3f admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.858981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba7c3-9d35-46bc-8dd9-d771e1ea4f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.859254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba7c3-9d35-46bc-8dd9-d771e1ea4f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.859940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ba7c3-9d35-46bc-8dd9-d771e1ea4f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:05.861562 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ba7c3-9d35-46bc-8dd9-d771e1ea4f3f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:05.872131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1324/2652] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:05 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:33:05.876516 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f470ffa-20cf-4857-b595-897183b80246 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:05.886340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f470ffa-20cf-4857-b595-897183b80246 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:05.891493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:05.892134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.892409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.892673 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] PATH_INFO: `/v3/projects/c4975cd9f3e34a55891787ad67e8eca7/users/111548f5b8a14ad9a68cd1b5730a7b38/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.907210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4975cd9f3e34a55891787ad67e8eca7, user_id=111548f5b8a14ad9a68cd1b5730a7b38, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:05.909843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:05.923358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1329/2653] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:33:05 2022] PUT /identity/v3/projects/c4975cd9f3e34a55891787ad67e8eca7/users/111548f5b8a14ad9a68cd1b5730a7b38/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:05.928398 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956be88-8ba0-4aca-9995-3c353cfbc5ed None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:05.928689 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956be88-8ba0-4aca-9995-3c353cfbc5ed None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:05.928966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956be88-8ba0-4aca-9995-3c353cfbc5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:05.948286 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-d956be88-8ba0-4aca-9995-3c353cfbc5ed None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:05.949537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d956be88-8ba0-4aca-9995-3c353cfbc5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:05.969491 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:06.011613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1325/2654] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:06.022252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:06.023935 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:06.085488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb876bd-00a1-41dc-b67b-b0c5a1d4d989 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:06.089033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f9d52-01c9-4558-bcb8-b5e724805036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:06.089674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f9d52-01c9-4558-bcb8-b5e724805036 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:06.089955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f9d52-01c9-4558-bcb8-b5e724805036 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:06.090518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f9d52-01c9-4558-bcb8-b5e724805036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:06.100133 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f9d52-01c9-4558-bcb8-b5e724805036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:06.101604 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f9d52-01c9-4558-bcb8-b5e724805036 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:06.111722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1330/2655] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:33:06.729202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b1ee4-0d08-48c4-9689-a73b326a1e30 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:06.729834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b1ee4-0d08-48c4-9689-a73b326a1e30 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:06.729834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b1ee4-0d08-48c4-9689-a73b326a1e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:06.759355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-560b1ee4-0d08-48c4-9689-a73b326a1e30 None None] MFA Rules not processed for user `111548f5b8a14ad9a68cd1b5730a7b38`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:06.760603 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-560b1ee4-0d08-48c4-9689-a73b326a1e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:06.827940 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1326/2656] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:06.832401 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f9d52-01c9-4558-bcb8-b5e724805036 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:06.842865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f9d52-01c9-4558-bcb8-b5e724805036 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:06.847392 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40b7e9-9c5f-4dad-9c22-b80113a8d817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:06.847988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40b7e9-9c5f-4dad-9c22-b80113a8d817 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:06.848261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40b7e9-9c5f-4dad-9c22-b80113a8d817 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:06.848524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40b7e9-9c5f-4dad-9c22-b80113a8d817 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:06.849173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40b7e9-9c5f-4dad-9c22-b80113a8d817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:06.851074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40b7e9-9c5f-4dad-9c22-b80113a8d817 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:06.877355 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1331/2657] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:33:06 2022] POST /identity/v3/projects => generated 374 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:33:06.881315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b1ee4-0d08-48c4-9689-a73b326a1e30 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:06.892377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b1ee4-0d08-48c4-9689-a73b326a1e30 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:06.948907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f6d90-e767-4b65-a0c4-792a7e6d7498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:06.949568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f6d90-e767-4b65-a0c4-792a7e6d7498 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:06.949846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f6d90-e767-4b65-a0c4-792a7e6d7498 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:06.950116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f6d90-e767-4b65-a0c4-792a7e6d7498 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:06.950810 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f6d90-e767-4b65-a0c4-792a7e6d7498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:06.952719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f6d90-e767-4b65-a0c4-792a7e6d7498 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:06.975152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1327/2658] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:33:06 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:33:06.979123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40b7e9-9c5f-4dad-9c22-b80113a8d817 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:06.989883 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40b7e9-9c5f-4dad-9c22-b80113a8d817 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:06.994326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4340ac-d236-4e0f-8c70-a72a253da47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:06.994926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4340ac-d236-4e0f-8c70-a72a253da47b admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:06.995202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4340ac-d236-4e0f-8c70-a72a253da47b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:06.995468 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4340ac-d236-4e0f-8c70-a72a253da47b admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:06.996148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4340ac-d236-4e0f-8c70-a72a253da47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:06.997779 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4340ac-d236-4e0f-8c70-a72a253da47b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:07.007949 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1332/2659] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:06 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:33:07.012100 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f6d90-e767-4b65-a0c4-792a7e6d7498 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:07.023220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f6d90-e767-4b65-a0c4-792a7e6d7498 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:07.027850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177dee9-415f-4220-8731-cda4034cb4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:07.029046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177dee9-415f-4220-8731-cda4034cb4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:07.029324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177dee9-415f-4220-8731-cda4034cb4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:07.029820 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177dee9-415f-4220-8731-cda4034cb4dd admin admin] PATH_INFO: `/v3/projects/80ec2de8688743b280b0d976e5965085/users/116f28d5c770491b9d62e37876e7b7b0/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:07.045152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177dee9-415f-4220-8731-cda4034cb4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ec2de8688743b280b0d976e5965085, user_id=116f28d5c770491b9d62e37876e7b7b0, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:07.047861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177dee9-415f-4220-8731-cda4034cb4dd admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:07.060572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1328/2660] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:33:07 2022] PUT /identity/v3/projects/80ec2de8688743b280b0d976e5965085/users/116f28d5c770491b9d62e37876e7b7b0/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:07.064453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4340ac-d236-4e0f-8c70-a72a253da47b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:07.074238 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4340ac-d236-4e0f-8c70-a72a253da47b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:07.085547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:07.135692 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f444d-58aa-423d-9515-e440c2219f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:07.136256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f444d-58aa-423d-9515-e440c2219f87 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:07.136538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f444d-58aa-423d-9515-e440c2219f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:07.136813 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f444d-58aa-423d-9515-e440c2219f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:07.137538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f444d-58aa-423d-9515-e440c2219f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:07.139658 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f444d-58aa-423d-9515-e440c2219f87 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:07.149396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1333/2661] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:07 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:33:07.154314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177dee9-415f-4220-8731-cda4034cb4dd admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:07.164698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177dee9-415f-4220-8731-cda4034cb4dd admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:07.169637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:07.170316 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:07.170599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:07.170875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] PATH_INFO: `/v3/projects/80ec2de8688743b280b0d976e5965085/users/116f28d5c770491b9d62e37876e7b7b0/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:07.172917 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ec2de8688743b280b0d976e5965085, user_id=116f28d5c770491b9d62e37876e7b7b0, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:07.175633 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:07.186034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1329/2662] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:33:07 2022] PUT /identity/v3/projects/80ec2de8688743b280b0d976e5965085/users/116f28d5c770491b9d62e37876e7b7b0/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:08.067155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171ed8-55cb-4d5c-91d4-42a12b7ef57c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:08.067691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171ed8-55cb-4d5c-91d4-42a12b7ef57c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:08.067691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171ed8-55cb-4d5c-91d4-42a12b7ef57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:08.096745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-51171ed8-55cb-4d5c-91d4-42a12b7ef57c None None] MFA Rules not processed for user `116f28d5c770491b9d62e37876e7b7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:08.098080 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-51171ed8-55cb-4d5c-91d4-42a12b7ef57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:08.173184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1334/2663] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:08.184437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:08.187067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:08.197167 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:08.251015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9ffb7-456f-4b03-9eb4-008000c7f253 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:08.254616 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e94c4-c437-4c74-a81e-cb9a2a404663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:08.255244 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e94c4-c437-4c74-a81e-cb9a2a404663 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:08.255523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e94c4-c437-4c74-a81e-cb9a2a404663 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:08.255944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e94c4-c437-4c74-a81e-cb9a2a404663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:08.311889 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e94c4-c437-4c74-a81e-cb9a2a404663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:08.313337 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e94c4-c437-4c74-a81e-cb9a2a404663 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:08.322210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1330/2664] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:09.670602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51171ed8-55cb-4d5c-91d4-42a12b7ef57c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:09.672016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-51171ed8-55cb-4d5c-91d4-42a12b7ef57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:09.683895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51171ed8-55cb-4d5c-91d4-42a12b7ef57c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:09.688260 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:09.688865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:09.689145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:09.689436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:09.698341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:09.699761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:09.709378 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1335/2665] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:33:15.647022 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e94c4-c437-4c74-a81e-cb9a2a404663 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:15.700806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e94c4-c437-4c74-a81e-cb9a2a404663 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:15.704352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:15.704962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:15.705236 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:15.705538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:15.707280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:15.765165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:15.766653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:15.775546 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1331/2666] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:19.168814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:19.170332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:19.260478 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f8432-f544-4c50-a3e1-6a4c66fec3bd service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:19.263640 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:19.264329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:19.264634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:19.264911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:19.266710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:19.278989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:19.280573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:19.291008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1336/2667] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:19 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:22.677093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:22.678589 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:22.692280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e6c3-fe13-410d-95a4-9108cadf23c5 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:22.696067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:22.696819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:22.697180 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:22.697554 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:22.700121 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:22.713472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:22.766726 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:22.768326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:22.778525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1332/2668] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:22.912631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:22.927011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4ee5-006d-4d59-bf38-383a8b36d826 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:22.932690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:22.933370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:22.933844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:22.934119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] PATH_INFO: `/v3/users/111548f5b8a14ad9a68cd1b5730a7b38` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:22.935414 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=111548f5b8a14ad9a68cd1b5730a7b38)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:22.937296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:22.939961 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:22.987324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111548f5b8a14ad9a68cd1b5730a7b38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:22.994143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111548f5b8a14ad9a68cd1b5730a7b38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.002914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111548f5b8a14ad9a68cd1b5730a7b38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.003605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111548f5b8a14ad9a68cd1b5730a7b38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.016140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111548f5b8a14ad9a68cd1b5730a7b38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.023223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1337/2669] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:33:22 2022] DELETE /identity/v3/users/111548f5b8a14ad9a68cd1b5730a7b38 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:23.207682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:23.219048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be587a5-5382-4e9b-8c59-832493047e6a service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:23.278145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:23.278792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:23.279068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:23.279338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] PATH_INFO: `/v3/projects/c4975cd9f3e34a55891787ad67e8eca7` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:23.280498 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4975cd9f3e34a55891787ad67e8eca7)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:23.282671 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:23.332507 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4975cd9f3e34a55891787ad67e8eca7', 'request_id': 'req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.340311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4975cd9f3e34a55891787ad67e8eca7', 'request_id': 'req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.346570 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4975cd9f3e34a55891787ad67e8eca7', 'request_id': 'req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.357523 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] The token cache is being invalidate because project c4975cd9f3e34a55891787ad67e8eca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:33:23.357806 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.358476 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.359980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1333/2670] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:33:23 2022] DELETE /identity/v3/projects/c4975cd9f3e34a55891787ad67e8eca7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:23.363981 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:23.365487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:23.379373 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818a51f-bfe3-4f33-8807-4fe28b650bd8 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:23.439291 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:23.439973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:23.440251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:23.440522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] PATH_INFO: `/v3/users/116f28d5c770491b9d62e37876e7b7b0` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:23.442016 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=116f28d5c770491b9d62e37876e7b7b0)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:23.443993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:23.498040 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116f28d5c770491b9d62e37876e7b7b0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.504936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116f28d5c770491b9d62e37876e7b7b0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.518840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116f28d5c770491b9d62e37876e7b7b0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.519598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116f28d5c770491b9d62e37876e7b7b0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.532289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-6d3f1131-f422-4130-9900-91a37f8c3288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116f28d5c770491b9d62e37876e7b7b0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.534446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:23.542230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1338/2671] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:33:23 2022] DELETE /identity/v3/users/116f28d5c770491b9d62e37876e7b7b0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:23.724028 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:23.726927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:23.765893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a9f87-68cb-4d9c-b3da-cdfb5a6e155d admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:23.811635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:23.835403 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:23.836104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:23.836385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:23.836653 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] PATH_INFO: `/v3/projects/80ec2de8688743b280b0d976e5965085` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:23.838526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ec2de8688743b280b0d976e5965085)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:23.842088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:23.900983 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ec2de8688743b280b0d976e5965085', 'request_id': 'req-9d764be5-cd53-438d-be1d-50d76657b26e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.909713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ec2de8688743b280b0d976e5965085', 'request_id': 'req-9d764be5-cd53-438d-be1d-50d76657b26e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.919423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ec2de8688743b280b0d976e5965085', 'request_id': 'req-9d764be5-cd53-438d-be1d-50d76657b26e'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.939186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] The token cache is being invalidate because project 80ec2de8688743b280b0d976e5965085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:33:23.939639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.940701 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-9d764be5-cd53-438d-be1d-50d76657b26e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:23.943383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1334/2672] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:33:23 2022] DELETE /identity/v3/projects/80ec2de8688743b280b0d976e5965085 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:23.948309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcebbcf-e0db-4989-8138-107ada32c902 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:23.948792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcebbcf-e0db-4989-8138-107ada32c902 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:23.949021 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcebbcf-e0db-4989-8138-107ada32c902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:23.969990 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5dcebbcf-e0db-4989-8138-107ada32c902 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:23.971261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5dcebbcf-e0db-4989-8138-107ada32c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:24.054081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1339/2673] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:24.062318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7479-2e66-47dc-ae05-a716bd981043 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.062678 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7479-2e66-47dc-ae05-a716bd981043 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.063031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7479-2e66-47dc-ae05-a716bd981043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.086946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-9cca7479-2e66-47dc-ae05-a716bd981043 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:24.088330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-9cca7479-2e66-47dc-ae05-a716bd981043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:24.097152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1335/2674] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:33:24.101594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcebbcf-e0db-4989-8138-107ada32c902 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:24.116508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcebbcf-e0db-4989-8138-107ada32c902 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:24.123844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:24.124633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.124972 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.125282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.126112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:33:24.126502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:24.128217 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:24.146925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca7479-2e66-47dc-ae05-a716bd981043 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:24.153846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1340/2675] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:33:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:33:24.164096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca7479-2e66-47dc-ae05-a716bd981043 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:24.169865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2be9c-fa29-49d9-8992-33b45471ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:24.171426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2be9c-fa29-49d9-8992-33b45471ae2f admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.171426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2be9c-fa29-49d9-8992-33b45471ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.171426 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2be9c-fa29-49d9-8992-33b45471ae2f admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.171708 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2be9c-fa29-49d9-8992-33b45471ae2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:24.173636 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2be9c-fa29-49d9-8992-33b45471ae2f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:24.204355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1336/2676] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:33:24 2022] POST /identity/v3/projects => generated 396 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:33:24.208676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:24.221486 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34143501-4a9e-492d-9182-10b59705f1c4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:24.333184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6165c03-6092-4514-9dae-8ee0df4c0df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:24.333991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6165c03-6092-4514-9dae-8ee0df4c0df4 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.334347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6165c03-6092-4514-9dae-8ee0df4c0df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.334700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6165c03-6092-4514-9dae-8ee0df4c0df4 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.335457 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6165c03-6092-4514-9dae-8ee0df4c0df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:24.337719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6165c03-6092-4514-9dae-8ee0df4c0df4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:24.370777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1341/2677] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:33:24 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:33:24.375697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2be9c-fa29-49d9-8992-33b45471ae2f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:24.387407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2be9c-fa29-49d9-8992-33b45471ae2f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:24.392515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971bdb-ce94-4d10-bbc3-1730fa10c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:24.393157 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971bdb-ce94-4d10-bbc3-1730fa10c42e admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.393465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971bdb-ce94-4d10-bbc3-1730fa10c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.393761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971bdb-ce94-4d10-bbc3-1730fa10c42e admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.394460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d971bdb-ce94-4d10-bbc3-1730fa10c42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:24.396093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d971bdb-ce94-4d10-bbc3-1730fa10c42e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:24.407443 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1337/2678] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:24 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:33:24.411729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6165c03-6092-4514-9dae-8ee0df4c0df4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:24.423600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6165c03-6092-4514-9dae-8ee0df4c0df4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:24.429283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32963b93-fb0e-40bb-aefd-7ba0727e635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:24.430139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32963b93-fb0e-40bb-aefd-7ba0727e635f admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.430430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32963b93-fb0e-40bb-aefd-7ba0727e635f admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.430703 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32963b93-fb0e-40bb-aefd-7ba0727e635f admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.431409 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32963b93-fb0e-40bb-aefd-7ba0727e635f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:24.433498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32963b93-fb0e-40bb-aefd-7ba0727e635f admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:24.446096 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1342/2679] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:24 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:33:24.451043 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971bdb-ce94-4d10-bbc3-1730fa10c42e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:24.461831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971bdb-ce94-4d10-bbc3-1730fa10c42e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:24.469383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:24.470190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.470490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.470825 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] PATH_INFO: `/v3/projects/3c3f129da82b445abf0cd3782d21d509/users/a51e73918bf749999d51008c3257444c/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.491705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c3f129da82b445abf0cd3782d21d509, user_id=a51e73918bf749999d51008c3257444c, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:24.494605 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:24.511433 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1338/2680] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:33:24 2022] PUT /identity/v3/projects/3c3f129da82b445abf0cd3782d21d509/users/a51e73918bf749999d51008c3257444c/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:24.522784 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb5e13-d791-45e9-8ade-bebe9a09ffcf None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.523264 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb5e13-d791-45e9-8ade-bebe9a09ffcf None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.523677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb5e13-d791-45e9-8ade-bebe9a09ffcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.553519 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-05cb5e13-d791-45e9-8ade-bebe9a09ffcf None None] MFA Rules not processed for user `a51e73918bf749999d51008c3257444c`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:24.554832 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-05cb5e13-d791-45e9-8ade-bebe9a09ffcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:24.614706 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:24.626033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1343/2681] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:24.637866 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:24.639715 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:24.712124 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3f51-d57d-4793-8eba-5c2f52255751 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:24.716547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4a88f-f8c4-4cb5-9fbf-b54f72e97962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:24.717190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4a88f-f8c4-4cb5-9fbf-b54f72e97962 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:24.717460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4a88f-f8c4-4cb5-9fbf-b54f72e97962 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:24.717737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4a88f-f8c4-4cb5-9fbf-b54f72e97962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:24.726949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4a88f-f8c4-4cb5-9fbf-b54f72e97962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:24.728499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4a88f-f8c4-4cb5-9fbf-b54f72e97962 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:24.738705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1339/2682] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:24 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:29.292623 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb5e13-d791-45e9-8ade-bebe9a09ffcf None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:29.294215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-05cb5e13-d791-45e9-8ade-bebe9a09ffcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:29.358989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb5e13-d791-45e9-8ade-bebe9a09ffcf None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:29.363424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:29.364052 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:29.364326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:29.364594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] PATH_INFO: `/v3/users/aaa106f1923b43dcb4ca4973815108c0` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:29.365760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa106f1923b43dcb4ca4973815108c0)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:29.367653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:29.413657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa106f1923b43dcb4ca4973815108c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.420929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa106f1923b43dcb4ca4973815108c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.427982 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa106f1923b43dcb4ca4973815108c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.428728 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa106f1923b43dcb4ca4973815108c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.441508 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa106f1923b43dcb4ca4973815108c0'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.447411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1344/2683] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:33:29 2022] DELETE /identity/v3/users/aaa106f1923b43dcb4ca4973815108c0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:29.624954 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4a88f-f8c4-4cb5-9fbf-b54f72e97962 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:29.636903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4a88f-f8c4-4cb5-9fbf-b54f72e97962 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:29.642379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:29.695992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:29.696632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:29.696910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:29.697193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] PATH_INFO: `/v3/projects/5bff3367439f45a7b7a420b66dd8574a` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:29.698347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bff3367439f45a7b7a420b66dd8574a)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:29.700794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:29.747207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bff3367439f45a7b7a420b66dd8574a', 'request_id': 'req-636617e9-f903-40cb-869d-13ccedf4c5d6'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.754087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bff3367439f45a7b7a420b66dd8574a', 'request_id': 'req-636617e9-f903-40cb-869d-13ccedf4c5d6'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.759612 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bff3367439f45a7b7a420b66dd8574a', 'request_id': 'req-636617e9-f903-40cb-869d-13ccedf4c5d6'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.771135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] The token cache is being invalidate because project 5bff3367439f45a7b7a420b66dd8574a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:33:29.771421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.772051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.773622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1340/2684] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:33:29 2022] DELETE /identity/v3/projects/5bff3367439f45a7b7a420b66dd8574a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:29.778336 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:29.779721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:29.792890 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0689c7-3833-4d4a-becb-a06a55e8bdf9 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:29.849018 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:29.849698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:29.849975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:29.850251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] PATH_INFO: `/v3/users/eb32acc0ef7f474fa2e6ccbd93fb2c38` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:29.851426 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb32acc0ef7f474fa2e6ccbd93fb2c38)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:29.853330 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:29.868142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:29.897610 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb32acc0ef7f474fa2e6ccbd93fb2c38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.905295 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb32acc0ef7f474fa2e6ccbd93fb2c38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.912490 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb32acc0ef7f474fa2e6ccbd93fb2c38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.913112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb32acc0ef7f474fa2e6ccbd93fb2c38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.924929 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb32acc0ef7f474fa2e6ccbd93fb2c38'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:29.930800 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1345/2685] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:33:29 2022] DELETE /identity/v3/users/eb32acc0ef7f474fa2e6ccbd93fb2c38 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:33:30.103098 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:30.104595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:30.120802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636617e9-f903-40cb-869d-13ccedf4c5d6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:30.177798 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:30.178437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:30.178714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:30.178985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] PATH_INFO: `/v3/projects/9ffc4f97d47d44d382250cb3a50e629e` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:30.180117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffc4f97d47d44d382250cb3a50e629e)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:30.182065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:30.222876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:30.229526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffc4f97d47d44d382250cb3a50e629e', 'request_id': 'req-32897680-6ea8-4b62-abcf-21d7098cb531'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:30.237434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffc4f97d47d44d382250cb3a50e629e', 'request_id': 'req-32897680-6ea8-4b62-abcf-21d7098cb531'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:30.243036 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffc4f97d47d44d382250cb3a50e629e', 'request_id': 'req-32897680-6ea8-4b62-abcf-21d7098cb531'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:30.254182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] The token cache is being invalidate because project 9ffc4f97d47d44d382250cb3a50e629e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:33:30.254471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:30.255105 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-32897680-6ea8-4b62-abcf-21d7098cb531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:30.256606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1341/2686] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:33:30 2022] DELETE /identity/v3/projects/9ffc4f97d47d44d382250cb3a50e629e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:33.910262 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:33.911698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:33.924316 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf276-17c6-4202-9d6d-6b382a35896b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:33.983375 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff625-4747-462d-b268-d966f034178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:33.984046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff625-4747-462d-b268-d966f034178b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:33.984324 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff625-4747-462d-b268-d966f034178b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:33.984594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff625-4747-462d-b268-d966f034178b admin admin] PATH_INFO: `/v3/users/a51e73918bf749999d51008c3257444c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:33.985808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ff625-4747-462d-b268-d966f034178b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a51e73918bf749999d51008c3257444c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:33.987773 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ff625-4747-462d-b268-d966f034178b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:34.037988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-844ff625-4747-462d-b268-d966f034178b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a51e73918bf749999d51008c3257444c'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.044986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-844ff625-4747-462d-b268-d966f034178b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a51e73918bf749999d51008c3257444c'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.052076 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-844ff625-4747-462d-b268-d966f034178b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a51e73918bf749999d51008c3257444c'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.052987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-844ff625-4747-462d-b268-d966f034178b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a51e73918bf749999d51008c3257444c'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.065241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-844ff625-4747-462d-b268-d966f034178b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a51e73918bf749999d51008c3257444c'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.071044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1346/2687] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:33:33 2022] DELETE /identity/v3/users/a51e73918bf749999d51008c3257444c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:34.076301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e001c30-2f05-45a5-a570-0dab0683242f None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:34.076795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e001c30-2f05-45a5-a570-0dab0683242f None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:34.076893 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e001c30-2f05-45a5-a570-0dab0683242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:34.095663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-5e001c30-2f05-45a5-a570-0dab0683242f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:34.096850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5e001c30-2f05-45a5-a570-0dab0683242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:34.158962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1342/2688] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:34.169470 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff625-4747-462d-b268-d966f034178b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:34.170895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-844ff625-4747-462d-b268-d966f034178b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:34.185576 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:34.231521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff625-4747-462d-b268-d966f034178b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:34.235170 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94dc7a-1315-491c-a752-6a4460f48946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:34.235772 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94dc7a-1315-491c-a752-6a4460f48946 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:34.236043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94dc7a-1315-491c-a752-6a4460f48946 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:34.236308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94dc7a-1315-491c-a752-6a4460f48946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:34.246043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94dc7a-1315-491c-a752-6a4460f48946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:34.247458 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94dc7a-1315-491c-a752-6a4460f48946 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:34.256670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1347/2689] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:34.529856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e001c30-2f05-45a5-a570-0dab0683242f None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:34.531299 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-5e001c30-2f05-45a5-a570-0dab0683242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:34.544905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e001c30-2f05-45a5-a570-0dab0683242f None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:34.549649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:34.550306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:34.550587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:34.550865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] PATH_INFO: `/v3/projects/3c3f129da82b445abf0cd3782d21d509` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:34.552026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c3f129da82b445abf0cd3782d21d509)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:34.554002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:34.726663 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c3f129da82b445abf0cd3782d21d509', 'request_id': 'req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.734495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c3f129da82b445abf0cd3782d21d509', 'request_id': 'req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.740048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c3f129da82b445abf0cd3782d21d509', 'request_id': 'req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.751075 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] The token cache is being invalidate because project 3c3f129da82b445abf0cd3782d21d509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:33:34.751366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.752000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-aa449e1b-ec53-4b8d-9958-7d3dc69b3873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:33:34.753525 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1343/2690] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:33:34 2022] DELETE /identity/v3/projects/3c3f129da82b445abf0cd3782d21d509 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:34.759566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a819a-e06c-4fad-9093-9a5da3ea9b40 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:34.759852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a819a-e06c-4fad-9093-9a5da3ea9b40 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:34.760121 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a819a-e06c-4fad-9093-9a5da3ea9b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:34.779522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-a26a819a-e06c-4fad-9093-9a5da3ea9b40 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:34.780704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a26a819a-e06c-4fad-9093-9a5da3ea9b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:34.837390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1348/2691] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:34.901026 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991674f-6232-422f-8969-ff12858f48f5 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:34.901467 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991674f-6232-422f-8969-ff12858f48f5 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:34.901780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991674f-6232-422f-8969-ff12858f48f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:34.920474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a991674f-6232-422f-8969-ff12858f48f5 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:34.921702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a991674f-6232-422f-8969-ff12858f48f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:34.929544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1344/2692] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:34.934185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a819a-e06c-4fad-9093-9a5da3ea9b40 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:34.946983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a819a-e06c-4fad-9093-9a5da3ea9b40 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:34.951674 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:34.952273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:34.952547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:34.953334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:34.954125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:33:34.954420 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:34.956004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:34.968863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1349/2693] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:33:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:33:34.973144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991674f-6232-422f-8969-ff12858f48f5 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:34.982434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991674f-6232-422f-8969-ff12858f48f5 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:34.987006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddc652-6533-407d-97f1-ee1fe0e6a9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:34.987602 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddc652-6533-407d-97f1-ee1fe0e6a9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:34.987875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddc652-6533-407d-97f1-ee1fe0e6a9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:34.988140 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddc652-6533-407d-97f1-ee1fe0e6a9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:34.988799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddc652-6533-407d-97f1-ee1fe0e6a9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:34.990705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddc652-6533-407d-97f1-ee1fe0e6a9a5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:35.017859 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1345/2694] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:33:34 2022] POST /identity/v3/projects => generated 380 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:33:35.022074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:35.032356 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57969937-d325-43b2-996c-88ce987032f6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:35.087858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc0bb8-ce3f-46f3-a0dc-a1c641d321d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:35.088491 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc0bb8-ce3f-46f3-a0dc-a1c641d321d0 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:35.088993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc0bb8-ce3f-46f3-a0dc-a1c641d321d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:35.089258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc0bb8-ce3f-46f3-a0dc-a1c641d321d0 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:35.090090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc0bb8-ce3f-46f3-a0dc-a1c641d321d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:35.091978 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc0bb8-ce3f-46f3-a0dc-a1c641d321d0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:35.117081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1350/2695] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:33:35 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:33:35.121714 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddc652-6533-407d-97f1-ee1fe0e6a9a5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:35.131899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddc652-6533-407d-97f1-ee1fe0e6a9a5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:35.136987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7754bf0-e0df-4433-8db3-cc3b04c0bc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:35.137611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7754bf0-e0df-4433-8db3-cc3b04c0bc74 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:35.137883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7754bf0-e0df-4433-8db3-cc3b04c0bc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:35.138149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7754bf0-e0df-4433-8db3-cc3b04c0bc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:35.138819 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7754bf0-e0df-4433-8db3-cc3b04c0bc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:35.140468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7754bf0-e0df-4433-8db3-cc3b04c0bc74 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:35.150113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1346/2696] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:35 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:33:35.154512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc0bb8-ce3f-46f3-a0dc-a1c641d321d0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:35.164461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc0bb8-ce3f-46f3-a0dc-a1c641d321d0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:35.169884 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9ac3e-dbfa-497b-a426-9544c5e58271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:35.170487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ac3e-dbfa-497b-a426-9544c5e58271 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:35.170769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ac3e-dbfa-497b-a426-9544c5e58271 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:35.171072 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ac3e-dbfa-497b-a426-9544c5e58271 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:35.171760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9ac3e-dbfa-497b-a426-9544c5e58271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:35.173383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9ac3e-dbfa-497b-a426-9544c5e58271 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:35.183987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1351/2697] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:35 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:33:35.187932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7754bf0-e0df-4433-8db3-cc3b04c0bc74 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:35.197791 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7754bf0-e0df-4433-8db3-cc3b04c0bc74 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:35.202617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:35.203258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:35.203531 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:35.203871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] PATH_INFO: `/v3/projects/49576f2166cc4b7f9261bd7cbc8384f4/users/43e543cfa3e1402e9694dcb095a26c00/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:35.215311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:35.219772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49576f2166cc4b7f9261bd7cbc8384f4, user_id=43e543cfa3e1402e9694dcb095a26c00, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:35.222996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:35.236828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1347/2698] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:33:35 2022] PUT /identity/v3/projects/49576f2166cc4b7f9261bd7cbc8384f4/users/43e543cfa3e1402e9694dcb095a26c00/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:35.241766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58b1e3-8a97-4aa5-b080-3ae6899c7bda None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:35.242053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58b1e3-8a97-4aa5-b080-3ae6899c7bda None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:35.243338 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58b1e3-8a97-4aa5-b080-3ae6899c7bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:35.262204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5e58b1e3-8a97-4aa5-b080-3ae6899c7bda None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:35.263418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5e58b1e3-8a97-4aa5-b080-3ae6899c7bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:35.280159 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:35.323992 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1352/2699] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:35.333614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:35.335331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:35.392906 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e77b2-90d6-4a5b-b96c-681440710d42 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:35.396591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd5ccc-eaec-4598-bda4-dcf1b5689b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:35.397204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd5ccc-eaec-4598-bda4-dcf1b5689b84 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:35.397514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd5ccc-eaec-4598-bda4-dcf1b5689b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:35.397789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd5ccc-eaec-4598-bda4-dcf1b5689b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:35.407464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd5ccc-eaec-4598-bda4-dcf1b5689b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:35.408895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd5ccc-eaec-4598-bda4-dcf1b5689b84 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:35.418213 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1348/2700] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:39.627269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa92d45-6450-447e-ab4c-30adf1fd62eb None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:39.627860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa92d45-6450-447e-ab4c-30adf1fd62eb None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:39.627860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa92d45-6450-447e-ab4c-30adf1fd62eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:39.659585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-0aa92d45-6450-447e-ab4c-30adf1fd62eb None None] MFA Rules not processed for user `43e543cfa3e1402e9694dcb095a26c00`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:39.660799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0aa92d45-6450-447e-ab4c-30adf1fd62eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:39.750134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1353/2701] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:39.754548 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd5ccc-eaec-4598-bda4-dcf1b5689b84 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:39.768801 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd5ccc-eaec-4598-bda4-dcf1b5689b84 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:39.774888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6068d16-376e-4666-b16c-98a7a4b0f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:39.775444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6068d16-376e-4666-b16c-98a7a4b0f191 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:39.776333 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6068d16-376e-4666-b16c-98a7a4b0f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:39.776606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6068d16-376e-4666-b16c-98a7a4b0f191 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:39.777334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6068d16-376e-4666-b16c-98a7a4b0f191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:39.779285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6068d16-376e-4666-b16c-98a7a4b0f191 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:39.809361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1349/2702] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:33:39 2022] POST /identity/v3/projects => generated 380 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:33:39.813305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa92d45-6450-447e-ab4c-30adf1fd62eb None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:39.824711 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa92d45-6450-447e-ab4c-30adf1fd62eb None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:39.886326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbade7f-9489-4c1d-93b2-0a09fce300d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:39.887027 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbade7f-9489-4c1d-93b2-0a09fce300d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:39.887300 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbade7f-9489-4c1d-93b2-0a09fce300d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:39.887566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbade7f-9489-4c1d-93b2-0a09fce300d1 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:39.888234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbade7f-9489-4c1d-93b2-0a09fce300d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:39.890176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbade7f-9489-4c1d-93b2-0a09fce300d1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:39.912908 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1354/2703] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:33:39 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:33:39.916925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6068d16-376e-4666-b16c-98a7a4b0f191 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:39.927631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6068d16-376e-4666-b16c-98a7a4b0f191 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:39.932221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a367bd-dfd3-4cb1-af13-44b00d070539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:39.932969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a367bd-dfd3-4cb1-af13-44b00d070539 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:39.933202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a367bd-dfd3-4cb1-af13-44b00d070539 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:39.933499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a367bd-dfd3-4cb1-af13-44b00d070539 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:39.934219 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a367bd-dfd3-4cb1-af13-44b00d070539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:39.935794 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a367bd-dfd3-4cb1-af13-44b00d070539 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:39.946808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1350/2704] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:39 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:33:39.951942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbade7f-9489-4c1d-93b2-0a09fce300d1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:39.963053 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbade7f-9489-4c1d-93b2-0a09fce300d1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:39.968665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a0c60-4d96-4297-90dd-c71c2dff620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:39.969311 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0c60-4d96-4297-90dd-c71c2dff620a admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:39.969615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0c60-4d96-4297-90dd-c71c2dff620a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:39.969899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0c60-4d96-4297-90dd-c71c2dff620a admin admin] PATH_INFO: `/v3/projects/a54de9fb6e9e4d6880ae34c5b3779635/users/c8a0ea459aad47f18d7e03f546f123cd/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:39.985680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a0c60-4d96-4297-90dd-c71c2dff620a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54de9fb6e9e4d6880ae34c5b3779635, user_id=c8a0ea459aad47f18d7e03f546f123cd, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:39.988407 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a0c60-4d96-4297-90dd-c71c2dff620a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:40.002783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1355/2705] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:33:39 2022] PUT /identity/v3/projects/a54de9fb6e9e4d6880ae34c5b3779635/users/c8a0ea459aad47f18d7e03f546f123cd/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:40.006993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a367bd-dfd3-4cb1-af13-44b00d070539 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:40.018160 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a367bd-dfd3-4cb1-af13-44b00d070539 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:40.077831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7403b-769a-49e2-ad7f-c7b0a691e18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:40.078472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7403b-769a-49e2-ad7f-c7b0a691e18f admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:40.078848 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7403b-769a-49e2-ad7f-c7b0a691e18f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:40.079119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7403b-769a-49e2-ad7f-c7b0a691e18f admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:40.079828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7403b-769a-49e2-ad7f-c7b0a691e18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:40.081492 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7403b-769a-49e2-ad7f-c7b0a691e18f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:40.096174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1351/2706] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:33:40 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:33:40.097722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a0c60-4d96-4297-90dd-c71c2dff620a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:40.108871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a0c60-4d96-4297-90dd-c71c2dff620a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:40.114318 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:40.114993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:40.115269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:40.115534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] PATH_INFO: `/v3/projects/a54de9fb6e9e4d6880ae34c5b3779635/users/c8a0ea459aad47f18d7e03f546f123cd/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:40.117512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54de9fb6e9e4d6880ae34c5b3779635, user_id=c8a0ea459aad47f18d7e03f546f123cd, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:40.120166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:40.142014 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1356/2707] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:33:40 2022] PUT /identity/v3/projects/a54de9fb6e9e4d6880ae34c5b3779635/users/c8a0ea459aad47f18d7e03f546f123cd/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:33:44.636543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe248da-047f-4c0b-beb2-583153d143f1 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:44.637149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe248da-047f-4c0b-beb2-583153d143f1 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:44.637149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe248da-047f-4c0b-beb2-583153d143f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:44.678538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-ebe248da-047f-4c0b-beb2-583153d143f1 None None] MFA Rules not processed for user `c8a0ea459aad47f18d7e03f546f123cd`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:44.679867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ebe248da-047f-4c0b-beb2-583153d143f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:44.760292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:44.803628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1352/2708] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:44.817974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:44.819913 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:44.827440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:33:44.915791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45630308-8d9e-4f2b-90d3-40dde495a30d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:44.922265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8e28f-24db-4df0-95ad-ade67c3165ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:44.922882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e28f-24db-4df0-95ad-ade67c3165ad service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:44.923160 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e28f-24db-4df0-95ad-ade67c3165ad service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:44.923611 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e28f-24db-4df0-95ad-ade67c3165ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:44.983351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8e28f-24db-4df0-95ad-ade67c3165ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:44.984937 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8e28f-24db-4df0-95ad-ade67c3165ad service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:44.994681 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1357/2709] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:48.185990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe248da-047f-4c0b-beb2-583153d143f1 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:48.188488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-ebe248da-047f-4c0b-beb2-583153d143f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:48.294650 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe248da-047f-4c0b-beb2-583153d143f1 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:48.300560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:48.301284 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:48.301576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:48.301852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:48.303607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:48.315905 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:48.317522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:48.328099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1353/2710] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:48 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:51.526944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2197-fdcd-46cd-ac32-4615ba1492e7 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:51.527440 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2197-fdcd-46cd-ac32-4615ba1492e7 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:51.527538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2197-fdcd-46cd-ac32-4615ba1492e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:51.546725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e12d2197-fdcd-46cd-ac32-4615ba1492e7 None None] MFA Rules not processed for user `890f3e80ae37401f8dad15d3d084f091`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:33:51.548130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e12d2197-fdcd-46cd-ac32-4615ba1492e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:51.607107 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1358/2711] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:33:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:33:51.924295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:51.925883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:51.994481 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9e9f-0553-425a-ace0-ca9064534bb2 service placement] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:51.998054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2e809-5685-4538-9b65-13947e98fcd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:51.998662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2e809-5685-4538-9b65-13947e98fcd9 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:51.998936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2e809-5685-4538-9b65-13947e98fcd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:51.999225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2e809-5685-4538-9b65-13947e98fcd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:52.000948 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-63f2e809-5685-4538-9b65-13947e98fcd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:52.014351 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2e809-5685-4538-9b65-13947e98fcd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:52.015825 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2e809-5685-4538-9b65-13947e98fcd9 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:52.024864 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1354/2712] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:51 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:33:58.160057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d2197-fdcd-46cd-ac32-4615ba1492e7 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:33:58.161835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e12d2197-fdcd-46cd-ac32-4615ba1492e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:58.179873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d2197-fdcd-46cd-ac32-4615ba1492e7 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:33:58.187394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:33:58.188103 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:33:58.188450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:33:58.188787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:33:58.190735 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:33:58.203186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:33:58.204768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:33:58.215725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1359/2713] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:33:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:34:02.020804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=67 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:34:06.712349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0215-c9b3-4a7d-850a-65c4e826039c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:34:06.712929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0215-c9b3-4a7d-850a-65c4e826039c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:34:06.712929 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0215-c9b3-4a7d-850a-65c4e826039c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:34:06.731547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-86bc0215-c9b3-4a7d-850a-65c4e826039c None None] MFA Rules not processed for user `43e543cfa3e1402e9694dcb095a26c00`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:34:06.732735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-86bc0215-c9b3-4a7d-850a-65c4e826039c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:34:06.739434 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1355/2714] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:34:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:34:08.209821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=85 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:34:16.726174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:34:24.150344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:34:24.160563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b67e1-1133-45a4-8ea9-fe62d6389a79 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:34:24.164474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:34:24.165088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:34:24.165369 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:34:24.165678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:34:24.174889 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:34:24.176335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:34:24.186552 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1360/2715] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:34:24 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:34:34.182346 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=6 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:17.055005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0215-c9b3-4a7d-850a-65c4e826039c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:17.056404 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-86bc0215-c9b3-4a7d-850a-65c4e826039c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:17.113535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0215-c9b3-4a7d-850a-65c4e826039c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:17.117091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:17.117737 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:17.118017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:17.118285 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:17.119996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:17.133808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:17.135248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:17.146759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1356/2716] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:17 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:26.394181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:26.395925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:26.408146 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa82f73-8375-4b13-b7e6-d8ac1138cc8f service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:26.411721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:26.412326 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:26.412616 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:26.412887 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:26.414633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:26.428031 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:26.429463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:26.440899 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1361/2717] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:26 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:27.141249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=26 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:36.435532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=6 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:47.480985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:47.490642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364216dd-dcda-42b5-92e8-c3b8faf45616 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:47.494142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:47.494739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:47.495013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:47.495280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:47.496994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:47.559591 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:47.561019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:47.571120 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1357/2718] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:50.689830 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:50.691407 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:50.707701 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71cf84-a1d8-4eff-b81c-49b9385ec0fc service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:50.712698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:50.713472 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:50.713846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:50.714115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] PATH_INFO: `/v3/users/890f3e80ae37401f8dad15d3d084f091` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:50.715347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=890f3e80ae37401f8dad15d3d084f091)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:50.717624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:50.768264 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '890f3e80ae37401f8dad15d3d084f091'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:50.775488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '890f3e80ae37401f8dad15d3d084f091'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:50.783782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '890f3e80ae37401f8dad15d3d084f091'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:50.784512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '890f3e80ae37401f8dad15d3d084f091'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:50.798505 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-69047fad-eba7-4c87-94b6-77a97b338c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '890f3e80ae37401f8dad15d3d084f091'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:50.804533 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1362/2719] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:35:50 2022] DELETE /identity/v3/users/890f3e80ae37401f8dad15d3d084f091 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:50.985297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:50.996709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e964a8-3285-4950-ac36-4e140d2cfd8c service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:51.055551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:51.056280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.056565 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.056832 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] PATH_INFO: `/v3/projects/477c661a08dd46b2a20f53a7944361ef` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.058009 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=477c661a08dd46b2a20f53a7944361ef)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:51.059892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:51.109990 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477c661a08dd46b2a20f53a7944361ef', 'request_id': 'req-de1c62cd-4e93-4e8d-850a-00180585aa03'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:51.117632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477c661a08dd46b2a20f53a7944361ef', 'request_id': 'req-de1c62cd-4e93-4e8d-850a-00180585aa03'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:51.123172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477c661a08dd46b2a20f53a7944361ef', 'request_id': 'req-de1c62cd-4e93-4e8d-850a-00180585aa03'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:51.135104 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] The token cache is being invalidate because project 477c661a08dd46b2a20f53a7944361ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:35:51.135419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:51.136076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-de1c62cd-4e93-4e8d-850a-00180585aa03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:51.137672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1358/2720] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:35:50 2022] DELETE /identity/v3/projects/477c661a08dd46b2a20f53a7944361ef => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:51.143386 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5cd2a-5285-4c80-9ae6-3a74f9ee881e None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.143766 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5cd2a-5285-4c80-9ae6-3a74f9ee881e None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.144049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5cd2a-5285-4c80-9ae6-3a74f9ee881e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.163315 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f1b5cd2a-5285-4c80-9ae6-3a74f9ee881e None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:51.164501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f1b5cd2a-5285-4c80-9ae6-3a74f9ee881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:51.227925 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1363/2721] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:51.235418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce04c3e-4f4c-4dc9-969f-b5ea60199e60 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.235710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce04c3e-4f4c-4dc9-969f-b5ea60199e60 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.235987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce04c3e-4f4c-4dc9-969f-b5ea60199e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.256269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-3ce04c3e-4f4c-4dc9-969f-b5ea60199e60 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:51.257534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3ce04c3e-4f4c-4dc9-969f-b5ea60199e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:51.266294 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1359/2722] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:51.270554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5cd2a-5285-4c80-9ae6-3a74f9ee881e None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:51.280531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5cd2a-5285-4c80-9ae6-3a74f9ee881e None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:51.284960 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:51.285618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.285900 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.286183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.286865 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:35:51.287174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:51.288777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:51.300879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1364/2723] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:35:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:35:51.305211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce04c3e-4f4c-4dc9-969f-b5ea60199e60 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:51.316030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce04c3e-4f4c-4dc9-969f-b5ea60199e60 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:51.320757 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e2c9-1ca5-41b8-8a41-dd0432516edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:51.321360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e2c9-1ca5-41b8-8a41-dd0432516edc admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.321681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e2c9-1ca5-41b8-8a41-dd0432516edc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.321946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e2c9-1ca5-41b8-8a41-dd0432516edc admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.322609 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0e2c9-1ca5-41b8-8a41-dd0432516edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:51.324490 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0e2c9-1ca5-41b8-8a41-dd0432516edc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:51.340292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:51.354849 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1360/2724] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:35:51 2022] POST /identity/v3/projects => generated 376 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:35:51.360227 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:51.370140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e4f68-6cdf-458f-bcb5-2a6541f4a1fb admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:51.424543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ca6ed-2af0-4fe1-a64e-b1659ddda621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:51.425150 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ca6ed-2af0-4fe1-a64e-b1659ddda621 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.425456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ca6ed-2af0-4fe1-a64e-b1659ddda621 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.425726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ca6ed-2af0-4fe1-a64e-b1659ddda621 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.426389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ca6ed-2af0-4fe1-a64e-b1659ddda621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:51.428268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ca6ed-2af0-4fe1-a64e-b1659ddda621 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:51.435966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:51.450597 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1365/2725] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:35:51 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:35:51.455984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e2c9-1ca5-41b8-8a41-dd0432516edc admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:51.466535 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e2c9-1ca5-41b8-8a41-dd0432516edc admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:51.471189 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116235de-bcf0-48bb-8f83-c36301df01fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:51.471793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116235de-bcf0-48bb-8f83-c36301df01fe admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.472073 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116235de-bcf0-48bb-8f83-c36301df01fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.472346 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116235de-bcf0-48bb-8f83-c36301df01fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.473037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116235de-bcf0-48bb-8f83-c36301df01fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:51.474659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116235de-bcf0-48bb-8f83-c36301df01fe admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:51.485331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1361/2726] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:51 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:51.489057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ca6ed-2af0-4fe1-a64e-b1659ddda621 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:51.499106 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ca6ed-2af0-4fe1-a64e-b1659ddda621 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:51.504796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c32d1-586a-4eee-83c7-a3da41acc753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:51.505396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c32d1-586a-4eee-83c7-a3da41acc753 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.505727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c32d1-586a-4eee-83c7-a3da41acc753 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.505983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c32d1-586a-4eee-83c7-a3da41acc753 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.506670 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c32d1-586a-4eee-83c7-a3da41acc753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:51.508237 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c32d1-586a-4eee-83c7-a3da41acc753 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:51.520049 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1366/2727] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:51 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:51.524166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116235de-bcf0-48bb-8f83-c36301df01fe admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:51.534128 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116235de-bcf0-48bb-8f83-c36301df01fe admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:51.538857 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:51.539547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.539828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.540106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] PATH_INFO: `/v3/projects/ca463520f5b944f9adf111b2a46e0219/users/9f5dd4aedaca4ad39979dfe294c87b40/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.555200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca463520f5b944f9adf111b2a46e0219, user_id=9f5dd4aedaca4ad39979dfe294c87b40, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:51.557869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:51.570783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1362/2728] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:51 2022] PUT /identity/v3/projects/ca463520f5b944f9adf111b2a46e0219/users/9f5dd4aedaca4ad39979dfe294c87b40/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:51.583320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b737ad3-d7df-4141-a0b5-99b0f4b6a940 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.583607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b737ad3-d7df-4141-a0b5-99b0f4b6a940 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.583911 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b737ad3-d7df-4141-a0b5-99b0f4b6a940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.611754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-9b737ad3-d7df-4141-a0b5-99b0f4b6a940 None None] MFA Rules not processed for user `9f5dd4aedaca4ad39979dfe294c87b40`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:51.613003 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-9b737ad3-d7df-4141-a0b5-99b0f4b6a940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:51.676722 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1367/2729] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:51.688549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:51.690257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:51.767099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ededf3c-6bf0-425d-a166-d7eabef3969c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:51.772478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb010c5-cc10-4159-90ff-a1d6f6d04992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:51.773087 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb010c5-cc10-4159-90ff-a1d6f6d04992 service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:51.773363 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb010c5-cc10-4159-90ff-a1d6f6d04992 service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:51.773662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb010c5-cc10-4159-90ff-a1d6f6d04992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:51.781318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb010c5-cc10-4159-90ff-a1d6f6d04992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:51.782944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb010c5-cc10-4159-90ff-a1d6f6d04992 service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:51.791627 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1363/2730] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:52.013188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b737ad3-d7df-4141-a0b5-99b0f4b6a940 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:52.076340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b737ad3-d7df-4141-a0b5-99b0f4b6a940 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:52.081296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:52.082225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:52.082502 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:52.082771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] PATH_INFO: `/v3/users/9f5dd4aedaca4ad39979dfe294c87b40` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:52.083912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5dd4aedaca4ad39979dfe294c87b40)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:52.085851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:52.131244 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5dd4aedaca4ad39979dfe294c87b40'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.137851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5dd4aedaca4ad39979dfe294c87b40'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.145762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5dd4aedaca4ad39979dfe294c87b40'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.146561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5dd4aedaca4ad39979dfe294c87b40'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.151696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:52.159864 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5dd4aedaca4ad39979dfe294c87b40'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.166172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1368/2731] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:35:52 2022] DELETE /identity/v3/users/9f5dd4aedaca4ad39979dfe294c87b40 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:52.170637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c261e2-8472-49bd-b2fb-fa87f4704acc None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:52.171154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c261e2-8472-49bd-b2fb-fa87f4704acc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:52.171154 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c261e2-8472-49bd-b2fb-fa87f4704acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:52.189928 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-71c261e2-8472-49bd-b2fb-fa87f4704acc None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:52.191110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-71c261e2-8472-49bd-b2fb-fa87f4704acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:52.251709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1364/2732] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:52.261384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:52.262839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:52.322690 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635479-d413-4444-8f1a-11e4cc52ab01 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:52.326286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe1664-3929-44b5-aa89-cd2da4a7d0c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:52.326948 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe1664-3929-44b5-aa89-cd2da4a7d0c3 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:52.327174 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe1664-3929-44b5-aa89-cd2da4a7d0c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:52.327441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe1664-3929-44b5-aa89-cd2da4a7d0c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:52.338592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe1664-3929-44b5-aa89-cd2da4a7d0c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:52.340050 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe1664-3929-44b5-aa89-cd2da4a7d0c3 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:52.349765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1369/2733] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:52.618377 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c261e2-8472-49bd-b2fb-fa87f4704acc None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:52.619911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-71c261e2-8472-49bd-b2fb-fa87f4704acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:52.633058 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c261e2-8472-49bd-b2fb-fa87f4704acc None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:52.637793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:52.638441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:52.638717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:52.638985 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] PATH_INFO: `/v3/projects/ca463520f5b944f9adf111b2a46e0219` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:52.640200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca463520f5b944f9adf111b2a46e0219)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:52.642276 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:52.684225 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:52.691269 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca463520f5b944f9adf111b2a46e0219', 'request_id': 'req-214a093c-85dd-40a0-b497-ad0f9196e780'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.698538 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca463520f5b944f9adf111b2a46e0219', 'request_id': 'req-214a093c-85dd-40a0-b497-ad0f9196e780'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.704397 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca463520f5b944f9adf111b2a46e0219', 'request_id': 'req-214a093c-85dd-40a0-b497-ad0f9196e780'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.715464 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] The token cache is being invalidate because project ca463520f5b944f9adf111b2a46e0219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:35:52.715746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.716381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-214a093c-85dd-40a0-b497-ad0f9196e780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:52.717949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1365/2734] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:35:52 2022] DELETE /identity/v3/projects/ca463520f5b944f9adf111b2a46e0219 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:52.723861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713fa3-401e-4220-b490-da39b8930c3f None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:52.724151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713fa3-401e-4220-b490-da39b8930c3f None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:52.724417 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713fa3-401e-4220-b490-da39b8930c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:52.743892 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-7e713fa3-401e-4220-b490-da39b8930c3f None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:52.745069 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7e713fa3-401e-4220-b490-da39b8930c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:52.804140 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1370/2735] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:52.811986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d2779-3679-45ac-a4ca-79dd5fc60963 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:52.812323 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d2779-3679-45ac-a4ca-79dd5fc60963 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:52.812598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d2779-3679-45ac-a4ca-79dd5fc60963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:52.832204 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-081d2779-3679-45ac-a4ca-79dd5fc60963 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:52.833447 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-081d2779-3679-45ac-a4ca-79dd5fc60963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:52.841175 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1366/2736] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:52.846013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e713fa3-401e-4220-b490-da39b8930c3f None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:52.855683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e713fa3-401e-4220-b490-da39b8930c3f None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:52.861012 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:52.861596 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:52.861877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:52.862148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:52.862863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:35:52.863165 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:52.864750 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:52.877479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1371/2737] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:35:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:35:52.881669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d2779-3679-45ac-a4ca-79dd5fc60963 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:52.891146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d2779-3679-45ac-a4ca-79dd5fc60963 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:52.895918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b486a-65ed-469a-9e54-282cda1dcbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:52.896583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b486a-65ed-469a-9e54-282cda1dcbc3 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:52.896861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b486a-65ed-469a-9e54-282cda1dcbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:52.897135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b486a-65ed-469a-9e54-282cda1dcbc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:52.897839 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b486a-65ed-469a-9e54-282cda1dcbc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:52.899761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b486a-65ed-469a-9e54-282cda1dcbc3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:52.929029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1367/2738] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:35:52 2022] POST /identity/v3/projects => generated 406 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:35:52.933081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:52.942941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2efb5-ecfd-48bc-a092-453def98c1f0 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.006994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c464e5d-5c5f-4038-8e6a-0b4841745d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.008085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c464e5d-5c5f-4038-8e6a-0b4841745d5e admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.008556 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c464e5d-5c5f-4038-8e6a-0b4841745d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.008977 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c464e5d-5c5f-4038-8e6a-0b4841745d5e admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.009997 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c464e5d-5c5f-4038-8e6a-0b4841745d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.011926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c464e5d-5c5f-4038-8e6a-0b4841745d5e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.033915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1372/2739] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:35:52 2022] POST /identity/v3/users => generated 304 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:53.038080 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b486a-65ed-469a-9e54-282cda1dcbc3 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.049320 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b486a-65ed-469a-9e54-282cda1dcbc3 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.054807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05c2c7-44b1-4817-85b8-0543efd99ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.055872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05c2c7-44b1-4817-85b8-0543efd99ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.056144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05c2c7-44b1-4817-85b8-0543efd99ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.056405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05c2c7-44b1-4817-85b8-0543efd99ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.057086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05c2c7-44b1-4817-85b8-0543efd99ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.059567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05c2c7-44b1-4817-85b8-0543efd99ea8 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.071517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1368/2740] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:53 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:53.075566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c464e5d-5c5f-4038-8e6a-0b4841745d5e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.085916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c464e5d-5c5f-4038-8e6a-0b4841745d5e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.090697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1b75d-4a34-4e3b-b274-d1eeba690393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.091305 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1b75d-4a34-4e3b-b274-d1eeba690393 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.091587 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1b75d-4a34-4e3b-b274-d1eeba690393 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.091863 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1b75d-4a34-4e3b-b274-d1eeba690393 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.092553 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1b75d-4a34-4e3b-b274-d1eeba690393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.095329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1b75d-4a34-4e3b-b274-d1eeba690393 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.096704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:53.107754 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1373/2741] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:53 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:53.112441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05c2c7-44b1-4817-85b8-0543efd99ea8 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.122456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05c2c7-44b1-4817-85b8-0543efd99ea8 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.127765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93a85d-a647-41e4-b249-35bbb685f38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.128416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93a85d-a647-41e4-b249-35bbb685f38c admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.128697 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93a85d-a647-41e4-b249-35bbb685f38c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.128966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93a85d-a647-41e4-b249-35bbb685f38c admin admin] PATH_INFO: `/v3/projects/6960e87236d84918aaf0113bce272364/users/9374aa616c2341bb901b9822c15ffa8d/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.143708 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93a85d-a647-41e4-b249-35bbb685f38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6960e87236d84918aaf0113bce272364, user_id=9374aa616c2341bb901b9822c15ffa8d, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.146437 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93a85d-a647-41e4-b249-35bbb685f38c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.159034 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1369/2742] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:53 2022] PUT /identity/v3/projects/6960e87236d84918aaf0113bce272364/users/9374aa616c2341bb901b9822c15ffa8d/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:53.169245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc53123-defb-4bca-ba5f-698279ab21a8 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.169589 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc53123-defb-4bca-ba5f-698279ab21a8 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.169868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc53123-defb-4bca-ba5f-698279ab21a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.198268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3cc53123-defb-4bca-ba5f-698279ab21a8 None None] MFA Rules not processed for user `9374aa616c2341bb901b9822c15ffa8d`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:53.199554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3cc53123-defb-4bca-ba5f-698279ab21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:53.262578 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1374/2743] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:53 2022] POST /identity/v3/auth/tokens => generated 3292 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:53.267626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93a85d-a647-41e4-b249-35bbb685f38c admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.328516 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93a85d-a647-41e4-b249-35bbb685f38c admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.351288 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7e56c-b9ca-4676-9da1-7fd634ab4a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.351941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e56c-b9ca-4676-9da1-7fd634ab4a35 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.352224 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e56c-b9ca-4676-9da1-7fd634ab4a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.352506 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e56c-b9ca-4676-9da1-7fd634ab4a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.353185 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7e56c-b9ca-4676-9da1-7fd634ab4a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.357397 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7e56c-b9ca-4676-9da1-7fd634ab4a35 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.385267 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1370/2744] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:35:53 2022] POST /identity/v3/projects => generated 404 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:53.389288 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc53123-defb-4bca-ba5f-698279ab21a8 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.400847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc53123-defb-4bca-ba5f-698279ab21a8 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.464099 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc606c-b6f6-4612-967c-695dab46de20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.464712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc606c-b6f6-4612-967c-695dab46de20 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.464988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc606c-b6f6-4612-967c-695dab46de20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.465255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc606c-b6f6-4612-967c-695dab46de20 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.465952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc606c-b6f6-4612-967c-695dab46de20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.467843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc606c-b6f6-4612-967c-695dab46de20 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.490314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1375/2745] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:35:53 2022] POST /identity/v3/users => generated 309 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:53.494300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7e56c-b9ca-4676-9da1-7fd634ab4a35 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.505792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7e56c-b9ca-4676-9da1-7fd634ab4a35 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.510720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842b8ee-3289-436d-a145-937452f31749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.511336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b8ee-3289-436d-a145-937452f31749 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.511617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b8ee-3289-436d-a145-937452f31749 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.511891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b8ee-3289-436d-a145-937452f31749 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.512585 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842b8ee-3289-436d-a145-937452f31749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.514226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842b8ee-3289-436d-a145-937452f31749 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.525329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1371/2746] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:53 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:53.529396 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc606c-b6f6-4612-967c-695dab46de20 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.540366 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc606c-b6f6-4612-967c-695dab46de20 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.544941 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718db88-e250-4fcb-858d-2e7fc91985b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.545684 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718db88-e250-4fcb-858d-2e7fc91985b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.545962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718db88-e250-4fcb-858d-2e7fc91985b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.547009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718db88-e250-4fcb-858d-2e7fc91985b6 admin admin] PATH_INFO: `/v3/projects/15805462e10d452e99bd3fded66b30f9/users/9a3c6cd545ca4d7989f541186f604c39/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.564946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718db88-e250-4fcb-858d-2e7fc91985b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15805462e10d452e99bd3fded66b30f9, user_id=9a3c6cd545ca4d7989f541186f604c39, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.567588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718db88-e250-4fcb-858d-2e7fc91985b6 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.582819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1376/2747] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:53 2022] PUT /identity/v3/projects/15805462e10d452e99bd3fded66b30f9/users/9a3c6cd545ca4d7989f541186f604c39/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:53.586913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842b8ee-3289-436d-a145-937452f31749 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.597822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842b8ee-3289-436d-a145-937452f31749 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.635223 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:53.655468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3e880-fc1d-48c8-a58a-0aab8f2176aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.656149 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3e880-fc1d-48c8-a58a-0aab8f2176aa admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.656423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3e880-fc1d-48c8-a58a-0aab8f2176aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.656698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3e880-fc1d-48c8-a58a-0aab8f2176aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.657375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3e880-fc1d-48c8-a58a-0aab8f2176aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.658987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3e880-fc1d-48c8-a58a-0aab8f2176aa admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.668446 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1372/2748] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:53 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:53.673006 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718db88-e250-4fcb-858d-2e7fc91985b6 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.683009 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718db88-e250-4fcb-858d-2e7fc91985b6 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.688047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.688683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.688954 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.689218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] PATH_INFO: `/v3/projects/15805462e10d452e99bd3fded66b30f9/users/9a3c6cd545ca4d7989f541186f604c39/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.691173 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15805462e10d452e99bd3fded66b30f9, user_id=9a3c6cd545ca4d7989f541186f604c39, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.693808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.703795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1377/2749] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:53 2022] PUT /identity/v3/projects/15805462e10d452e99bd3fded66b30f9/users/9a3c6cd545ca4d7989f541186f604c39/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:53.715161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde15b70-f9f0-4ea0-9cf5-23b3266d3e06 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.715472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde15b70-f9f0-4ea0-9cf5-23b3266d3e06 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.715746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde15b70-f9f0-4ea0-9cf5-23b3266d3e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.742725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-fde15b70-f9f0-4ea0-9cf5-23b3266d3e06 None None] MFA Rules not processed for user `9a3c6cd545ca4d7989f541186f604c39`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:53.743910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fde15b70-f9f0-4ea0-9cf5-23b3266d3e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:53.808885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1373/2750] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:53 2022] POST /identity/v3/auth/tokens => generated 3357 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:53.820013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:53.821789 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:53.864397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:53.880402 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a4671-5c67-41cd-be04-c5a2be0986f4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:53.884054 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d2141-a4c4-47fb-bf3e-7521373faea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:53.884663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2141-a4c4-47fb-bf3e-7521373faea4 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:53.884947 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2141-a4c4-47fb-bf3e-7521373faea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:53.885214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2141-a4c4-47fb-bf3e-7521373faea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:53.894798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2141-a4c4-47fb-bf3e-7521373faea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:53.896216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2141-a4c4-47fb-bf3e-7521373faea4 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:53.905591 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1378/2751] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:53 2022] GET /identity/v3/auth/tokens => generated 3357 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:54.052807 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde15b70-f9f0-4ea0-9cf5-23b3266d3e06 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:54.115182 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde15b70-f9f0-4ea0-9cf5-23b3266d3e06 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:54.119670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:54.120305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:54.120586 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:54.120853 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] PATH_INFO: `/v3/users/9374aa616c2341bb901b9822c15ffa8d` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:54.122025 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9374aa616c2341bb901b9822c15ffa8d)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:54.123944 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:54.172067 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9374aa616c2341bb901b9822c15ffa8d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.179378 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9374aa616c2341bb901b9822c15ffa8d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.180006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9374aa616c2341bb901b9822c15ffa8d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.187805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:54.194626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9374aa616c2341bb901b9822c15ffa8d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.195375 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9374aa616c2341bb901b9822c15ffa8d'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.208822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1374/2752] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:35:54 2022] DELETE /identity/v3/users/9374aa616c2341bb901b9822c15ffa8d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:54.213287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf292da-2819-4b53-be14-cf3b626e56fa None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:54.213824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf292da-2819-4b53-be14-cf3b626e56fa None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:54.213943 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf292da-2819-4b53-be14-cf3b626e56fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:54.232815 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-fdf292da-2819-4b53-be14-cf3b626e56fa None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:54.234366 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fdf292da-2819-4b53-be14-cf3b626e56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:54.292768 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1379/2753] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:54.302657 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:54.304044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:54.362090 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86306992-bfab-4d1e-837c-af4113a04e3a admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:54.365734 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79220002-f2e6-4424-986c-979fe63f99fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:54.366350 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79220002-f2e6-4424-986c-979fe63f99fc service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:54.366634 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79220002-f2e6-4424-986c-979fe63f99fc service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:54.366910 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79220002-f2e6-4424-986c-979fe63f99fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:54.376510 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79220002-f2e6-4424-986c-979fe63f99fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:54.378008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79220002-f2e6-4424-986c-979fe63f99fc service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:54.387563 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1375/2754] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:54.659136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf292da-2819-4b53-be14-cf3b626e56fa None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:54.660603 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fdf292da-2819-4b53-be14-cf3b626e56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:54.673022 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf292da-2819-4b53-be14-cf3b626e56fa None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:54.677619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:54.678256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:54.678628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:54.678802 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] PATH_INFO: `/v3/projects/6960e87236d84918aaf0113bce272364` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:54.680038 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6960e87236d84918aaf0113bce272364)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:54.681906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:54.729901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6960e87236d84918aaf0113bce272364', 'request_id': 'req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.735570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6960e87236d84918aaf0113bce272364', 'request_id': 'req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.743538 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6960e87236d84918aaf0113bce272364', 'request_id': 'req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.754424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] The token cache is being invalidate because project 6960e87236d84918aaf0113bce272364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:35:54.754710 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.755335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.756843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1380/2755] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:35:54 2022] DELETE /identity/v3/projects/6960e87236d84918aaf0113bce272364 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:54.760808 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79220002-f2e6-4424-986c-979fe63f99fc service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:54.771143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79220002-f2e6-4424-986c-979fe63f99fc service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:54.828212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:54.828836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:54.829110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:54.829376 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] PATH_INFO: `/v3/users/9a3c6cd545ca4d7989f541186f604c39` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:54.830599 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a3c6cd545ca4d7989f541186f604c39)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:54.832742 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:54.877300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a3c6cd545ca4d7989f541186f604c39'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.884564 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a3c6cd545ca4d7989f541186f604c39'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.885187 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a3c6cd545ca4d7989f541186f604c39'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.897353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a3c6cd545ca4d7989f541186f604c39'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.898029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fd6583ce-202f-4e10-9046-296f75060f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a3c6cd545ca4d7989f541186f604c39'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:54.910383 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1376/2756] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:35:54 2022] DELETE /identity/v3/users/9a3c6cd545ca4d7989f541186f604c39 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:55.185290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:55.187097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:55.201774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa22d-1339-40c2-92cf-4e6b5cebda8a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:55.260280 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:55.260926 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.261117 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.261390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] PATH_INFO: `/v3/projects/15805462e10d452e99bd3fded66b30f9` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.262566 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15805462e10d452e99bd3fded66b30f9)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:55.264461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:55.271197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:55.313136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15805462e10d452e99bd3fded66b30f9', 'request_id': 'req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:55.318479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15805462e10d452e99bd3fded66b30f9', 'request_id': 'req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:55.325700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15805462e10d452e99bd3fded66b30f9', 'request_id': 'req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:55.338184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] The token cache is being invalidate because project 15805462e10d452e99bd3fded66b30f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:35:55.339013 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:55.339286 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-82b80172-a6f0-4ad0-a6d4-6a9c6d682b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:55.340668 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1381/2757] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:35:55 2022] DELETE /identity/v3/projects/15805462e10d452e99bd3fded66b30f9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:55.346631 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfa449-cba8-497b-bf3c-ca6d003814e6 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.347139 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfa449-cba8-497b-bf3c-ca6d003814e6 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.347254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfa449-cba8-497b-bf3c-ca6d003814e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.354782 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:55.368766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-08bfa449-cba8-497b-bf3c-ca6d003814e6 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:55.370015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-08bfa449-cba8-497b-bf3c-ca6d003814e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:55.433953 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1377/2758] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:55.441903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b34c25-0e78-42e7-8dc4-862cdc0ce48c None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.442185 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b34c25-0e78-42e7-8dc4-862cdc0ce48c None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.442456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b34c25-0e78-42e7-8dc4-862cdc0ce48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.462816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-90b34c25-0e78-42e7-8dc4-862cdc0ce48c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:55.464945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-90b34c25-0e78-42e7-8dc4-862cdc0ce48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:55.476918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1382/2759] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:55.481336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfa449-cba8-497b-bf3c-ca6d003814e6 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:55.493249 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfa449-cba8-497b-bf3c-ca6d003814e6 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:55.497913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:55.498556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.498829 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.499095 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.499895 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:35:55.500194 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:55.501925 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:55.515950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1378/2760] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:35:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:35:55.520426 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b34c25-0e78-42e7-8dc4-862cdc0ce48c None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:55.539278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b34c25-0e78-42e7-8dc4-862cdc0ce48c None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:55.549436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a4e6-a509-43d6-bc3e-1a6b1679fdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:55.550351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a4e6-a509-43d6-bc3e-1a6b1679fdff admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.550633 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a4e6-a509-43d6-bc3e-1a6b1679fdff admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.550905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a4e6-a509-43d6-bc3e-1a6b1679fdff admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.551599 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20a4e6-a509-43d6-bc3e-1a6b1679fdff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:55.555329 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20a4e6-a509-43d6-bc3e-1a6b1679fdff admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:55.592511 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1383/2761] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:35:55 2022] POST /identity/v3/projects => generated 364 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:35:55.597292 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:55.607526 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664c2c-1012-4a61-bd8b-36492c90ca80 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:55.664328 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95ea7-8772-409d-9168-562df1b6ef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:55.664951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95ea7-8772-409d-9168-562df1b6ef16 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.665263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95ea7-8772-409d-9168-562df1b6ef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.665572 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95ea7-8772-409d-9168-562df1b6ef16 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.666240 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de95ea7-8772-409d-9168-562df1b6ef16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:55.668172 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de95ea7-8772-409d-9168-562df1b6ef16 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:55.690119 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1379/2762] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:35:55 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:55.694320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a4e6-a509-43d6-bc3e-1a6b1679fdff admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:55.706090 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a4e6-a509-43d6-bc3e-1a6b1679fdff admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:55.710881 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee827a-c082-4f9d-9e72-1bfab3609335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:55.711487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee827a-c082-4f9d-9e72-1bfab3609335 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.711762 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee827a-c082-4f9d-9e72-1bfab3609335 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.712030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee827a-c082-4f9d-9e72-1bfab3609335 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.712740 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee827a-c082-4f9d-9e72-1bfab3609335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:55.715592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee827a-c082-4f9d-9e72-1bfab3609335 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:55.725974 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1384/2763] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:55 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:55.730086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95ea7-8772-409d-9168-562df1b6ef16 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:55.741004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95ea7-8772-409d-9168-562df1b6ef16 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:55.745770 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71eb6e6-d1b0-4b3a-ae61-7e96843bceed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:55.746388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71eb6e6-d1b0-4b3a-ae61-7e96843bceed admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.746665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71eb6e6-d1b0-4b3a-ae61-7e96843bceed admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.746934 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71eb6e6-d1b0-4b3a-ae61-7e96843bceed admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.747621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71eb6e6-d1b0-4b3a-ae61-7e96843bceed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:55.749773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71eb6e6-d1b0-4b3a-ae61-7e96843bceed admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:55.764282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1380/2764] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:55 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:55.769178 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee827a-c082-4f9d-9e72-1bfab3609335 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:55.780554 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee827a-c082-4f9d-9e72-1bfab3609335 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:55.785283 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2beee7-8211-4a37-862b-b86b89818102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:55.786051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2beee7-8211-4a37-862b-b86b89818102 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.786331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2beee7-8211-4a37-862b-b86b89818102 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.786606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2beee7-8211-4a37-862b-b86b89818102 admin admin] PATH_INFO: `/v3/projects/d24b98c0669143459b265799d933aa9b/users/f61b51a339d343ddacbe1c2b8d328bc8/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.802729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2beee7-8211-4a37-862b-b86b89818102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d24b98c0669143459b265799d933aa9b, user_id=f61b51a339d343ddacbe1c2b8d328bc8, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:55.805718 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2beee7-8211-4a37-862b-b86b89818102 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:55.819854 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1385/2765] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:55 2022] PUT /identity/v3/projects/d24b98c0669143459b265799d933aa9b/users/f61b51a339d343ddacbe1c2b8d328bc8/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:55.832287 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018407a-fc1c-4275-9951-23333fd348d3 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:55.832569 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018407a-fc1c-4275-9951-23333fd348d3 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:55.832854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018407a-fc1c-4275-9951-23333fd348d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:55.865942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-d018407a-fc1c-4275-9951-23333fd348d3 None None] MFA Rules not processed for user `f61b51a339d343ddacbe1c2b8d328bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:55.867233 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-d018407a-fc1c-4275-9951-23333fd348d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:55.934389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:55.941161 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1381/2766] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:55.945314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2beee7-8211-4a37-862b-b86b89818102 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:56.017277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2beee7-8211-4a37-862b-b86b89818102 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:56.022312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95231ebd-5652-4ea8-bbfa-b6b1c452cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:56.022868 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231ebd-5652-4ea8-bbfa-b6b1c452cfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:56.023148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231ebd-5652-4ea8-bbfa-b6b1c452cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:56.023421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231ebd-5652-4ea8-bbfa-b6b1c452cfc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:56.024627 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95231ebd-5652-4ea8-bbfa-b6b1c452cfc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:56.026328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95231ebd-5652-4ea8-bbfa-b6b1c452cfc1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:56.054810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1386/2767] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:35:55 2022] POST /identity/v3/projects => generated 364 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:56.059008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018407a-fc1c-4275-9951-23333fd348d3 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:56.071987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018407a-fc1c-4275-9951-23333fd348d3 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:56.150911 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e6b3f-c693-4f6c-9411-74ba42f727e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:56.151541 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e6b3f-c693-4f6c-9411-74ba42f727e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:56.151821 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e6b3f-c693-4f6c-9411-74ba42f727e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:56.152093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e6b3f-c693-4f6c-9411-74ba42f727e5 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:56.152767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e6b3f-c693-4f6c-9411-74ba42f727e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:56.154852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e6b3f-c693-4f6c-9411-74ba42f727e5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:56.178920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1382/2768] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:35:56 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:56.182877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95231ebd-5652-4ea8-bbfa-b6b1c452cfc1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:56.195151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95231ebd-5652-4ea8-bbfa-b6b1c452cfc1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:56.200704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b873baf-edb1-4a7d-831e-27f391e3c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:56.201320 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873baf-edb1-4a7d-831e-27f391e3c3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:56.201697 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873baf-edb1-4a7d-831e-27f391e3c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:56.201967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873baf-edb1-4a7d-831e-27f391e3c3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:56.202788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b873baf-edb1-4a7d-831e-27f391e3c3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:56.204370 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b873baf-edb1-4a7d-831e-27f391e3c3d8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:56.217362 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1387/2769] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:56 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:56.221676 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e6b3f-c693-4f6c-9411-74ba42f727e5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:56.232199 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e6b3f-c693-4f6c-9411-74ba42f727e5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:56.239131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7878cb9-cada-47b2-bc97-b523fd9708e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:56.239804 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7878cb9-cada-47b2-bc97-b523fd9708e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:56.240084 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7878cb9-cada-47b2-bc97-b523fd9708e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:56.241949 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7878cb9-cada-47b2-bc97-b523fd9708e6 admin admin] PATH_INFO: `/v3/projects/e4eb53d469b345c288e1bcc605e5c038/users/a6241eefa88e48938231d1020a805945/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:56.258152 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7878cb9-cada-47b2-bc97-b523fd9708e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4eb53d469b345c288e1bcc605e5c038, user_id=a6241eefa88e48938231d1020a805945, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:56.261313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7878cb9-cada-47b2-bc97-b523fd9708e6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:56.276301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1383/2770] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:56 2022] PUT /identity/v3/projects/e4eb53d469b345c288e1bcc605e5c038/users/a6241eefa88e48938231d1020a805945/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:56.280232 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b873baf-edb1-4a7d-831e-27f391e3c3d8 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:56.287563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:56.291842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b873baf-edb1-4a7d-831e-27f391e3c3d8 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:56.358427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dd2a0-286b-4128-8082-df415314a0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:56.359677 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dd2a0-286b-4128-8082-df415314a0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:56.359995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dd2a0-286b-4128-8082-df415314a0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:56.360259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dd2a0-286b-4128-8082-df415314a0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:56.360939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dd2a0-286b-4128-8082-df415314a0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:56.362559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dd2a0-286b-4128-8082-df415314a0b3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:56.374148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1388/2771] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:56 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 22:35:56.378640 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7878cb9-cada-47b2-bc97-b523fd9708e6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:56.392885 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7878cb9-cada-47b2-bc97-b523fd9708e6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:56.399210 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:56.400179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:56.400456 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:56.400727 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] PATH_INFO: `/v3/projects/e4eb53d469b345c288e1bcc605e5c038/users/a6241eefa88e48938231d1020a805945/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:56.402861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4eb53d469b345c288e1bcc605e5c038, user_id=a6241eefa88e48938231d1020a805945, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:56.406057 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:56.417162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1384/2772] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:56 2022] PUT /identity/v3/projects/e4eb53d469b345c288e1bcc605e5c038/users/a6241eefa88e48938231d1020a805945/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:56.490258 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4026-5d0e-4fc2-bc70-8d7fcdb61f1d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:56.490536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4026-5d0e-4fc2-bc70-8d7fcdb61f1d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:56.490805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4026-5d0e-4fc2-bc70-8d7fcdb61f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:56.522171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-46cf4026-5d0e-4fc2-bc70-8d7fcdb61f1d None None] MFA Rules not processed for user `a6241eefa88e48938231d1020a805945`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:56.524263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-46cf4026-5d0e-4fc2-bc70-8d7fcdb61f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:56.620712 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1389/2773] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:56.632632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:56.634381 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:56.717018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71364ff-e5d3-4e5e-b37c-6ccf53716b43 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:56.721349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e865844-11c4-4e8b-a355-7b9fefd2d0be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:56.721997 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e865844-11c4-4e8b-a355-7b9fefd2d0be service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:56.722282 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e865844-11c4-4e8b-a355-7b9fefd2d0be service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:56.722556 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e865844-11c4-4e8b-a355-7b9fefd2d0be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:56.731551 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e865844-11c4-4e8b-a355-7b9fefd2d0be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:56.732986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e865844-11c4-4e8b-a355-7b9fefd2d0be service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:56.742533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1385/2774] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 22:35:57.158942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4026-5d0e-4fc2-bc70-8d7fcdb61f1d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:57.220169 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4026-5d0e-4fc2-bc70-8d7fcdb61f1d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:57.224745 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:57.225349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:57.225653 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:57.226033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] PATH_INFO: `/v3/users/f61b51a339d343ddacbe1c2b8d328bc8` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:57.227188 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f61b51a339d343ddacbe1c2b8d328bc8)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:57.229085 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:57.257989 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:57.276358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f61b51a339d343ddacbe1c2b8d328bc8'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.284112 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f61b51a339d343ddacbe1c2b8d328bc8'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.290973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f61b51a339d343ddacbe1c2b8d328bc8'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.291592 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f61b51a339d343ddacbe1c2b8d328bc8'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.308075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f61b51a339d343ddacbe1c2b8d328bc8'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.314876 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1390/2775] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:35:57 2022] DELETE /identity/v3/users/f61b51a339d343ddacbe1c2b8d328bc8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:57.320759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097adcf-0287-47a8-8853-154711ed9d8c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:57.321245 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097adcf-0287-47a8-8853-154711ed9d8c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:57.321783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097adcf-0287-47a8-8853-154711ed9d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:57.344091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0097adcf-0287-47a8-8853-154711ed9d8c None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:57.345344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0097adcf-0287-47a8-8853-154711ed9d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:57.389389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:57.402549 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1386/2776] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:57.412695 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:57.414219 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:57.472444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a84f37-f6bf-4c62-9e42-0cf896577c99 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:57.476040 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294f73a-bd42-46a1-8c01-8a9bb4676cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:57.476646 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f73a-bd42-46a1-8c01-8a9bb4676cfd service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:57.476923 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f73a-bd42-46a1-8c01-8a9bb4676cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:57.477194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f73a-bd42-46a1-8c01-8a9bb4676cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:57.487365 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294f73a-bd42-46a1-8c01-8a9bb4676cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:57.488786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294f73a-bd42-46a1-8c01-8a9bb4676cfd service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:57.498081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1391/2777] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:57.774761 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097adcf-0287-47a8-8853-154711ed9d8c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:57.776169 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0097adcf-0287-47a8-8853-154711ed9d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:57.789359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097adcf-0287-47a8-8853-154711ed9d8c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:57.794785 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:57.795647 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:57.795988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:57.796261 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] PATH_INFO: `/v3/projects/d24b98c0669143459b265799d933aa9b` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:57.797584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d24b98c0669143459b265799d933aa9b)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:57.799560 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:57.847637 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd24b98c0669143459b265799d933aa9b', 'request_id': 'req-fae27418-a059-48b5-8119-068f9cb20969'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.857088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd24b98c0669143459b265799d933aa9b', 'request_id': 'req-fae27418-a059-48b5-8119-068f9cb20969'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.863010 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd24b98c0669143459b265799d933aa9b', 'request_id': 'req-fae27418-a059-48b5-8119-068f9cb20969'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.873884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] The token cache is being invalidate because project d24b98c0669143459b265799d933aa9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:35:57.874722 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.875453 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.876942 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1387/2778] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:35:57 2022] DELETE /identity/v3/projects/d24b98c0669143459b265799d933aa9b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:57.880942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294f73a-bd42-46a1-8c01-8a9bb4676cfd service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:57.891467 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294f73a-bd42-46a1-8c01-8a9bb4676cfd service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:57.946993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:57.947543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:57.947819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:57.948086 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] PATH_INFO: `/v3/users/a6241eefa88e48938231d1020a805945` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:57.949224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6241eefa88e48938231d1020a805945)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:57.951158 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:57.997465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6241eefa88e48938231d1020a805945'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:57.999944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:58.005942 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6241eefa88e48938231d1020a805945'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.013648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6241eefa88e48938231d1020a805945'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.014262 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6241eefa88e48938231d1020a805945'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.027289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-bad67acb-d4cf-42f9-aaaa-cdeee5dacd9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6241eefa88e48938231d1020a805945'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.034196 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1392/2779] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:35:57 2022] DELETE /identity/v3/users/a6241eefa88e48938231d1020a805945 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:58.311902 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:58.313353 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:58.326966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27418-a059-48b5-8119-068f9cb20969 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:58.385312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:58.386030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.386313 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.386604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] PATH_INFO: `/v3/projects/e4eb53d469b345c288e1bcc605e5c038` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.387823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4eb53d469b345c288e1bcc605e5c038)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:58.390600 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:58.435869 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4eb53d469b345c288e1bcc605e5c038', 'request_id': 'req-16a62c78-6793-408b-b955-7dc938236132'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.443306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4eb53d469b345c288e1bcc605e5c038', 'request_id': 'req-16a62c78-6793-408b-b955-7dc938236132'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.448924 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4eb53d469b345c288e1bcc605e5c038', 'request_id': 'req-16a62c78-6793-408b-b955-7dc938236132'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.460776 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] The token cache is being invalidate because project e4eb53d469b345c288e1bcc605e5c038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:35:58.461065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.461716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-16a62c78-6793-408b-b955-7dc938236132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:58.463260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1388/2780] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:35:58 2022] DELETE /identity/v3/projects/e4eb53d469b345c288e1bcc605e5c038 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:35:58.468678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c5989-ebc1-4ba7-a565-44f2a11e9f26 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.469151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c5989-ebc1-4ba7-a565-44f2a11e9f26 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.469303 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c5989-ebc1-4ba7-a565-44f2a11e9f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.488795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-752c5989-ebc1-4ba7-a565-44f2a11e9f26 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:58.490030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-752c5989-ebc1-4ba7-a565-44f2a11e9f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:58.550079 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1393/2781] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:58.557632 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed933-dc3f-4740-8f6c-149b495146d2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.557918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed933-dc3f-4740-8f6c-149b495146d2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.558191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed933-dc3f-4740-8f6c-149b495146d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.572780 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:58.579532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-463ed933-dc3f-4740-8f6c-149b495146d2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:58.581301 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-463ed933-dc3f-4740-8f6c-149b495146d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:58.589812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1389/2782] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:58.594461 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c5989-ebc1-4ba7-a565-44f2a11e9f26 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:58.604184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c5989-ebc1-4ba7-a565-44f2a11e9f26 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:58.608987 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:58.609634 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.609915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.610190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.610878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:35:58.611182 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:58.612777 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:58.625224 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1394/2783] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:35:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 22:35:58.629297 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed933-dc3f-4740-8f6c-149b495146d2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:58.638515 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed933-dc3f-4740-8f6c-149b495146d2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:58.643835 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c6d5b-a8d9-4f35-9000-8421d4b6d854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:58.644479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6d5b-a8d9-4f35-9000-8421d4b6d854 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.644753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6d5b-a8d9-4f35-9000-8421d4b6d854 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.645018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6d5b-a8d9-4f35-9000-8421d4b6d854 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.645702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c6d5b-a8d9-4f35-9000-8421d4b6d854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:58.647579 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c6d5b-a8d9-4f35-9000-8421d4b6d854 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:58.673858 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1390/2784] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:35:58 2022] POST /identity/v3/projects => generated 388 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:58.677964 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:58.688350 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549744d-092c-4985-a4bf-2f35d9c20c41 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:58.745352 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefff2-c035-43c3-a2d1-e6b432cecfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:58.745921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefff2-c035-43c3-a2d1-e6b432cecfb3 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.746209 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefff2-c035-43c3-a2d1-e6b432cecfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.746481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefff2-c035-43c3-a2d1-e6b432cecfb3 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.747134 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deefff2-c035-43c3-a2d1-e6b432cecfb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:58.749903 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deefff2-c035-43c3-a2d1-e6b432cecfb3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:58.771061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1395/2785] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:35:58 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:58.775241 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c6d5b-a8d9-4f35-9000-8421d4b6d854 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:58.785517 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c6d5b-a8d9-4f35-9000-8421d4b6d854 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:58.790037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f609e-7126-47b5-8dd0-5940c0594eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:58.790635 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f609e-7126-47b5-8dd0-5940c0594eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.790908 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f609e-7126-47b5-8dd0-5940c0594eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.791174 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f609e-7126-47b5-8dd0-5940c0594eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.791852 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f609e-7126-47b5-8dd0-5940c0594eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:58.793499 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f609e-7126-47b5-8dd0-5940c0594eb5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:58.803280 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1391/2786] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:58 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:58.807051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefff2-c035-43c3-a2d1-e6b432cecfb3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:58.817993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefff2-c035-43c3-a2d1-e6b432cecfb3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:58.822585 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a89da5-c0b1-49d3-9aab-09f7506f2f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:58.823181 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89da5-c0b1-49d3-9aab-09f7506f2f71 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.823459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89da5-c0b1-49d3-9aab-09f7506f2f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.823729 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89da5-c0b1-49d3-9aab-09f7506f2f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.824420 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89da5-c0b1-49d3-9aab-09f7506f2f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:58.826040 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89da5-c0b1-49d3-9aab-09f7506f2f71 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:58.837775 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1396/2787] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:58 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:58.841792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f609e-7126-47b5-8dd0-5940c0594eb5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:58.851670 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f609e-7126-47b5-8dd0-5940c0594eb5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:58.856227 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5e9e1-c9ae-4185-8121-013179779f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:58.856872 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5e9e1-c9ae-4185-8121-013179779f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.857146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5e9e1-c9ae-4185-8121-013179779f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.857432 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5e9e1-c9ae-4185-8121-013179779f6f admin admin] PATH_INFO: `/v3/projects/513e0a79329142d4a0bdc4c87116d549/users/89bfadf55f264d9697ae70f9fad02e06/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.871960 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5e9e1-c9ae-4185-8121-013179779f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=513e0a79329142d4a0bdc4c87116d549, user_id=89bfadf55f264d9697ae70f9fad02e06, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:58.875168 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5e9e1-c9ae-4185-8121-013179779f6f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:58.887941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1392/2788] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:58 2022] PUT /identity/v3/projects/513e0a79329142d4a0bdc4c87116d549/users/89bfadf55f264d9697ae70f9fad02e06/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:35:58.900436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b89c3-05cd-4afd-ba95-226542235341 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:58.900723 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b89c3-05cd-4afd-ba95-226542235341 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:58.900998 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b89c3-05cd-4afd-ba95-226542235341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:58.928444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-5a7b89c3-05cd-4afd-ba95-226542235341 None None] MFA Rules not processed for user `89bfadf55f264d9697ae70f9fad02e06`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:58.929664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-5a7b89c3-05cd-4afd-ba95-226542235341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:58.949123 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:58.996180 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1397/2789] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:59.001713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5e9e1-c9ae-4185-8121-013179779f6f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:59.066293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5e9e1-c9ae-4185-8121-013179779f6f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:59.071303 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeebd9-60ff-451a-815b-82f2fd6550bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:59.071611 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeebd9-60ff-451a-815b-82f2fd6550bc admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.071888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeebd9-60ff-451a-815b-82f2fd6550bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.072156 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeebd9-60ff-451a-815b-82f2fd6550bc admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.072828 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdeebd9-60ff-451a-815b-82f2fd6550bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:59.074856 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdeebd9-60ff-451a-815b-82f2fd6550bc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:59.103477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1393/2790] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:35:59 2022] POST /identity/v3/projects => generated 390 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:59.107615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b89c3-05cd-4afd-ba95-226542235341 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:59.118782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b89c3-05cd-4afd-ba95-226542235341 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:59.178540 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08932f2c-fe4b-4877-bca2-b096fdd8935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:59.179171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932f2c-fe4b-4877-bca2-b096fdd8935b admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.179450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932f2c-fe4b-4877-bca2-b096fdd8935b admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.179805 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932f2c-fe4b-4877-bca2-b096fdd8935b admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.180465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08932f2c-fe4b-4877-bca2-b096fdd8935b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:59.182525 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08932f2c-fe4b-4877-bca2-b096fdd8935b admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:59.205895 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1398/2791] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:35:59 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:35:59.209978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeebd9-60ff-451a-815b-82f2fd6550bc admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:59.220822 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeebd9-60ff-451a-815b-82f2fd6550bc admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:59.225528 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfbd81-7fdb-42e3-831b-926cf4748d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:59.226134 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfbd81-7fdb-42e3-831b-926cf4748d1f admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.226410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfbd81-7fdb-42e3-831b-926cf4748d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.226683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfbd81-7fdb-42e3-831b-926cf4748d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.227407 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfbd81-7fdb-42e3-831b-926cf4748d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:59.229016 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfbd81-7fdb-42e3-831b-926cf4748d1f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:59.240305 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1394/2792] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:59 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:59.244253 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08932f2c-fe4b-4877-bca2-b096fdd8935b admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:59.255004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08932f2c-fe4b-4877-bca2-b096fdd8935b admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:59.259506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f3970-ba93-4acc-bddc-c7fad2d9d67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:59.260151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f3970-ba93-4acc-bddc-c7fad2d9d67d admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.260427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f3970-ba93-4acc-bddc-c7fad2d9d67d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.260698 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f3970-ba93-4acc-bddc-c7fad2d9d67d admin admin] PATH_INFO: `/v3/projects/45e167522c844badb91bf0576daa5a98/users/57dafb8bc19345d0b0bb4ec0669537e4/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.274909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f3970-ba93-4acc-bddc-c7fad2d9d67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e167522c844badb91bf0576daa5a98, user_id=57dafb8bc19345d0b0bb4ec0669537e4, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:59.277543 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f3970-ba93-4acc-bddc-c7fad2d9d67d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:59.290034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1399/2793] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:59 2022] PUT /identity/v3/projects/45e167522c844badb91bf0576daa5a98/users/57dafb8bc19345d0b0bb4ec0669537e4/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:59.294001 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfbd81-7fdb-42e3-831b-926cf4748d1f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:59.305732 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfbd81-7fdb-42e3-831b-926cf4748d1f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:59.363789 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd83502-bc3f-4036-92e0-4658be2adf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:59.364404 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83502-bc3f-4036-92e0-4658be2adf9b admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.364682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83502-bc3f-4036-92e0-4658be2adf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.364951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83502-bc3f-4036-92e0-4658be2adf9b admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.365669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd83502-bc3f-4036-92e0-4658be2adf9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:59.367257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd83502-bc3f-4036-92e0-4658be2adf9b admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:59.377135 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1395/2794] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:35:59 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:35:59.381261 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f3970-ba93-4acc-bddc-c7fad2d9d67d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:59.391810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f3970-ba93-4acc-bddc-c7fad2d9d67d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:59.396279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:59.396920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.397191 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.397492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] PATH_INFO: `/v3/projects/45e167522c844badb91bf0576daa5a98/users/57dafb8bc19345d0b0bb4ec0669537e4/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.399397 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e167522c844badb91bf0576daa5a98, user_id=57dafb8bc19345d0b0bb4ec0669537e4, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:59.402065 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:59.412512 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1400/2795] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:35:59 2022] PUT /identity/v3/projects/45e167522c844badb91bf0576daa5a98/users/57dafb8bc19345d0b0bb4ec0669537e4/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:59.423253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b595d-d7f1-4a5b-874c-91e1f9e832cc None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.423793 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b595d-d7f1-4a5b-874c-91e1f9e832cc None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.423945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b595d-d7f1-4a5b-874c-91e1f9e832cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.448527 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:59.451595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e57b595d-d7f1-4a5b-874c-91e1f9e832cc None None] MFA Rules not processed for user `57dafb8bc19345d0b0bb4ec0669537e4`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:59.453459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e57b595d-d7f1-4a5b-874c-91e1f9e832cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:59.522229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1396/2796] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:35:59.534066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:59.536048 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:59.593877 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d79d60-c7b7-4679-92f3-2975ce31ed67 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:59.597782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe0e5b-a411-49ba-8e11-6d79293f3624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:59.598388 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0e5b-a411-49ba-8e11-6d79293f3624 service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.598667 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0e5b-a411-49ba-8e11-6d79293f3624 service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.598939 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0e5b-a411-49ba-8e11-6d79293f3624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.607871 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe0e5b-a411-49ba-8e11-6d79293f3624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:59.609976 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe0e5b-a411-49ba-8e11-6d79293f3624 service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:59.619476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1401/2797] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:35:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:35:59.754665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b595d-d7f1-4a5b-874c-91e1f9e832cc None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:35:59.816126 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b595d-d7f1-4a5b-874c-91e1f9e832cc None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:35:59.820539 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:35:59.821166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.821465 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.821739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] PATH_INFO: `/v3/users/89bfadf55f264d9697ae70f9fad02e06` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.822897 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=89bfadf55f264d9697ae70f9fad02e06)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:35:59.824805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:35:59.872787 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89bfadf55f264d9697ae70f9fad02e06'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:59.880705 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89bfadf55f264d9697ae70f9fad02e06'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:59.881361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89bfadf55f264d9697ae70f9fad02e06'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:59.893209 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89bfadf55f264d9697ae70f9fad02e06'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:59.893878 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89bfadf55f264d9697ae70f9fad02e06'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:35:59.906428 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1397/2798] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:35:59 2022] DELETE /identity/v3/users/89bfadf55f264d9697ae70f9fad02e06 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:35:59.911142 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb507d-9871-4363-9d89-1e4bfae68529 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:35:59.911427 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb507d-9871-4363-9d89-1e4bfae68529 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:35:59.911693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb507d-9871-4363-9d89-1e4bfae68529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:35:59.930973 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-3adb507d-9871-4363-9d89-1e4bfae68529 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:35:59.932171 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3adb507d-9871-4363-9d89-1e4bfae68529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:35:59.944068 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:35:59.992866 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1402/2799] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:35:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:00.003278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:00.004786 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:00.024968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:00.064601 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52b8a5-bc35-49bc-a699-9ea3bbbad4bc admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:00.068332 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05542d-8bd4-4295-9298-f1aa9aafa637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:00.068901 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05542d-8bd4-4295-9298-f1aa9aafa637 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:00.069176 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05542d-8bd4-4295-9298-f1aa9aafa637 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:00.069478 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05542d-8bd4-4295-9298-f1aa9aafa637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:00.078643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05542d-8bd4-4295-9298-f1aa9aafa637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:00.080086 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05542d-8bd4-4295-9298-f1aa9aafa637 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:00.089777 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1398/2800] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:00.366241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb507d-9871-4363-9d89-1e4bfae68529 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:00.367642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-3adb507d-9871-4363-9d89-1e4bfae68529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:00.381851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb507d-9871-4363-9d89-1e4bfae68529 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:00.386411 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:00.387044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:00.387321 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:00.387586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] PATH_INFO: `/v3/projects/513e0a79329142d4a0bdc4c87116d549` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:00.389340 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=513e0a79329142d4a0bdc4c87116d549)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:00.391445 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:00.438840 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513e0a79329142d4a0bdc4c87116d549', 'request_id': 'req-d860cbcb-682c-4188-a31d-ff30421c58dc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.444282 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513e0a79329142d4a0bdc4c87116d549', 'request_id': 'req-d860cbcb-682c-4188-a31d-ff30421c58dc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.451590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513e0a79329142d4a0bdc4c87116d549', 'request_id': 'req-d860cbcb-682c-4188-a31d-ff30421c58dc'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.463544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] The token cache is being invalidate because project 513e0a79329142d4a0bdc4c87116d549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:36:00.463827 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.464446 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.465971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1403/2801] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:36:00 2022] DELETE /identity/v3/projects/513e0a79329142d4a0bdc4c87116d549 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:00.471315 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05542d-8bd4-4295-9298-f1aa9aafa637 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:00.481008 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05542d-8bd4-4295-9298-f1aa9aafa637 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:00.535471 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:00.536106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:00.536380 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:00.536649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] PATH_INFO: `/v3/users/57dafb8bc19345d0b0bb4ec0669537e4` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:00.537818 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57dafb8bc19345d0b0bb4ec0669537e4)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:00.539743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:00.584823 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57dafb8bc19345d0b0bb4ec0669537e4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.591903 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57dafb8bc19345d0b0bb4ec0669537e4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.593143 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57dafb8bc19345d0b0bb4ec0669537e4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.604947 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57dafb8bc19345d0b0bb4ec0669537e4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.605595 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-02ca0301-c9ac-4023-86d6-7292e5e02407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57dafb8bc19345d0b0bb4ec0669537e4'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:00.617971 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1399/2802] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:36:00 2022] DELETE /identity/v3/users/57dafb8bc19345d0b0bb4ec0669537e4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:00.895331 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:00.896946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:00.911798 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860cbcb-682c-4188-a31d-ff30421c58dc admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:00.969912 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:00.970613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:00.970945 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:00.971259 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] PATH_INFO: `/v3/projects/45e167522c844badb91bf0576daa5a98` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:00.972532 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e167522c844badb91bf0576daa5a98)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:00.974476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.018763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:01.023002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e167522c844badb91bf0576daa5a98', 'request_id': 'req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:01.030111 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e167522c844badb91bf0576daa5a98', 'request_id': 'req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:01.037279 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e167522c844badb91bf0576daa5a98', 'request_id': 'req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:01.048551 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] The token cache is being invalidate because project 45e167522c844badb91bf0576daa5a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:36:01.048837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:01.049488 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-51868c93-1a0c-41e6-8e70-1ee10f9ba2d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:01.050980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1404/2803] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:36:00 2022] DELETE /identity/v3/projects/45e167522c844badb91bf0576daa5a98 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:01.055946 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca4933-4c3e-43c5-9b94-bff6d7c4c5d2 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.056229 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca4933-4c3e-43c5-9b94-bff6d7c4c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.056503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca4933-4c3e-43c5-9b94-bff6d7c4c5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.076039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-42ca4933-4c3e-43c5-9b94-bff6d7c4c5d2 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:01.077874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-42ca4933-4c3e-43c5-9b94-bff6d7c4c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:01.112017 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:01.140683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1400/2804] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:01.148297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a6f0-b56d-47cc-9ebc-003e3d1488fa None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.148756 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a6f0-b56d-47cc-9ebc-003e3d1488fa None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.149030 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a6f0-b56d-47cc-9ebc-003e3d1488fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.168322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-f6c1a6f0-b56d-47cc-9ebc-003e3d1488fa None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:01.169544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f6c1a6f0-b56d-47cc-9ebc-003e3d1488fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:01.178875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1405/2805] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:01.182724 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca4933-4c3e-43c5-9b94-bff6d7c4c5d2 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.193270 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca4933-4c3e-43c5-9b94-bff6d7c4c5d2 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.197907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.198519 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.198803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.199079 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.199766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:36:01.200069 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.201698 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.214011 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1401/2806] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:36:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:36:01.218763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a6f0-b56d-47cc-9ebc-003e3d1488fa None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.227825 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a6f0-b56d-47cc-9ebc-003e3d1488fa None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.232384 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080d2ed-c438-489f-81f6-426ff8728529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.232980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080d2ed-c438-489f-81f6-426ff8728529 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.234418 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080d2ed-c438-489f-81f6-426ff8728529 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.234700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080d2ed-c438-489f-81f6-426ff8728529 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.235354 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080d2ed-c438-489f-81f6-426ff8728529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.237230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080d2ed-c438-489f-81f6-426ff8728529 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.265678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1406/2807] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:36:01 2022] POST /identity/v3/projects => generated 384 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:01.269783 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.279723 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e58fde-0b50-4bfd-9b35-256dae240829 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.336497 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff768835-71d8-4b93-a117-e4efaf7fb526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.337117 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff768835-71d8-4b93-a117-e4efaf7fb526 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.337399 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff768835-71d8-4b93-a117-e4efaf7fb526 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.337709 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff768835-71d8-4b93-a117-e4efaf7fb526 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.338361 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff768835-71d8-4b93-a117-e4efaf7fb526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.340290 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff768835-71d8-4b93-a117-e4efaf7fb526 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.362388 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1402/2808] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:36:01 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:01.366719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080d2ed-c438-489f-81f6-426ff8728529 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.377549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080d2ed-c438-489f-81f6-426ff8728529 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.382128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477c92f-84a4-47b2-bb28-7630b754578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.382727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c92f-84a4-47b2-bb28-7630b754578e admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.383005 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c92f-84a4-47b2-bb28-7630b754578e admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.383275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c92f-84a4-47b2-bb28-7630b754578e admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.384057 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477c92f-84a4-47b2-bb28-7630b754578e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.385833 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477c92f-84a4-47b2-bb28-7630b754578e admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.396466 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1407/2809] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:01 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:01.400035 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff768835-71d8-4b93-a117-e4efaf7fb526 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.411207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff768835-71d8-4b93-a117-e4efaf7fb526 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.415861 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f8436-8c08-4c8f-bbe0-4fb8c231138e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.416606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f8436-8c08-4c8f-bbe0-4fb8c231138e admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.416890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f8436-8c08-4c8f-bbe0-4fb8c231138e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.417165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f8436-8c08-4c8f-bbe0-4fb8c231138e admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.417900 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f8436-8c08-4c8f-bbe0-4fb8c231138e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.419488 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f8436-8c08-4c8f-bbe0-4fb8c231138e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.430981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1403/2810] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:01 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:01.435083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477c92f-84a4-47b2-bb28-7630b754578e admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.445761 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477c92f-84a4-47b2-bb28-7630b754578e admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.450358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a58e16-fcb5-470c-becd-2588f14adedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.451011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a58e16-fcb5-470c-becd-2588f14adedc admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.451290 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a58e16-fcb5-470c-becd-2588f14adedc admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.451563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a58e16-fcb5-470c-becd-2588f14adedc admin admin] PATH_INFO: `/v3/projects/457b91686be64992b991199419ef15aa/users/bd446fea256f4d8eb7c199b0d9e8acfb/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.466605 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a58e16-fcb5-470c-becd-2588f14adedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=457b91686be64992b991199419ef15aa, user_id=bd446fea256f4d8eb7c199b0d9e8acfb, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.469615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a58e16-fcb5-470c-becd-2588f14adedc admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.482855 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1408/2811] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:01 2022] PUT /identity/v3/projects/457b91686be64992b991199419ef15aa/users/bd446fea256f4d8eb7c199b0d9e8acfb/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:01.494921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00be9-b605-4dc3-9cce-4c47d8cfc053 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.495423 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00be9-b605-4dc3-9cce-4c47d8cfc053 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.495530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00be9-b605-4dc3-9cce-4c47d8cfc053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.523875 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-6be00be9-b605-4dc3-9cce-4c47d8cfc053 None None] MFA Rules not processed for user `bd446fea256f4d8eb7c199b0d9e8acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:01.525082 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6be00be9-b605-4dc3-9cce-4c47d8cfc053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:01.589379 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1404/2812] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:01.594601 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a58e16-fcb5-470c-becd-2588f14adedc admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.656967 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a58e16-fcb5-470c-becd-2588f14adedc admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.661438 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a23c26-da64-4f38-a49f-7b60bcfc8f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.662043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a23c26-da64-4f38-a49f-7b60bcfc8f11 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.662317 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a23c26-da64-4f38-a49f-7b60bcfc8f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.662584 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a23c26-da64-4f38-a49f-7b60bcfc8f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.663239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a23c26-da64-4f38-a49f-7b60bcfc8f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.665118 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a23c26-da64-4f38-a49f-7b60bcfc8f11 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.692213 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1409/2813] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:36:01 2022] POST /identity/v3/projects => generated 384 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:36:01.696421 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be00be9-b605-4dc3-9cce-4c47d8cfc053 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.708254 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be00be9-b605-4dc3-9cce-4c47d8cfc053 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.741982 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:01.769907 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a219-c3e5-47c4-a7c7-d8cda1ff7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.770532 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a219-c3e5-47c4-a7c7-d8cda1ff7a6e admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.770799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a219-c3e5-47c4-a7c7-d8cda1ff7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.771077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a219-c3e5-47c4-a7c7-d8cda1ff7a6e admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.771730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3a219-c3e5-47c4-a7c7-d8cda1ff7a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.773684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3a219-c3e5-47c4-a7c7-d8cda1ff7a6e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.793514 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1405/2814] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:36:01 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:01.797689 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a23c26-da64-4f38-a49f-7b60bcfc8f11 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.807966 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a23c26-da64-4f38-a49f-7b60bcfc8f11 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.813364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23821f18-afc8-4aa6-ac1f-aa0114696cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.813991 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23821f18-afc8-4aa6-ac1f-aa0114696cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.814265 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23821f18-afc8-4aa6-ac1f-aa0114696cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.814531 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23821f18-afc8-4aa6-ac1f-aa0114696cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.815210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23821f18-afc8-4aa6-ac1f-aa0114696cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.816812 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23821f18-afc8-4aa6-ac1f-aa0114696cf7 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.827819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1410/2815] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:01 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:01.832416 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a219-c3e5-47c4-a7c7-d8cda1ff7a6e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.842291 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a219-c3e5-47c4-a7c7-d8cda1ff7a6e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.846996 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0b78d-1afb-41c9-b747-bfbf6179d091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.847644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0b78d-1afb-41c9-b747-bfbf6179d091 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.847918 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0b78d-1afb-41c9-b747-bfbf6179d091 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.848186 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0b78d-1afb-41c9-b747-bfbf6179d091 admin admin] PATH_INFO: `/v3/projects/135a3a7067314226a38976efa00873b5/users/bb0b0b973a90468a93064e28bed91a48/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.862937 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0b78d-1afb-41c9-b747-bfbf6179d091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=135a3a7067314226a38976efa00873b5, user_id=bb0b0b973a90468a93064e28bed91a48, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.865684 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0b78d-1afb-41c9-b747-bfbf6179d091 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.878999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1406/2816] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:01 2022] PUT /identity/v3/projects/135a3a7067314226a38976efa00873b5/users/bb0b0b973a90468a93064e28bed91a48/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:01.882910 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23821f18-afc8-4aa6-ac1f-aa0114696cf7 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.893915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23821f18-afc8-4aa6-ac1f-aa0114696cf7 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.933721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:01.952088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02edd0e-1e65-4f81-88d5-5f478c3af5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.952699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02edd0e-1e65-4f81-88d5-5f478c3af5bb admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.952975 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02edd0e-1e65-4f81-88d5-5f478c3af5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.953245 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02edd0e-1e65-4f81-88d5-5f478c3af5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.953962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02edd0e-1e65-4f81-88d5-5f478c3af5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.955534 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02edd0e-1e65-4f81-88d5-5f478c3af5bb admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:01.965335 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1411/2817] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:01 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:01.969702 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0b78d-1afb-41c9-b747-bfbf6179d091 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:01.981890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0b78d-1afb-41c9-b747-bfbf6179d091 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:01.987200 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:01.987863 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:01.988138 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:01.988405 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] PATH_INFO: `/v3/projects/135a3a7067314226a38976efa00873b5/users/bb0b0b973a90468a93064e28bed91a48/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:01.990360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=135a3a7067314226a38976efa00873b5, user_id=bb0b0b973a90468a93064e28bed91a48, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:01.992969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:02.003888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1407/2818] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:01 2022] PUT /identity/v3/projects/135a3a7067314226a38976efa00873b5/users/bb0b0b973a90468a93064e28bed91a48/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:02.014394 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c50da-b0c1-46a9-b240-75ea72e59413 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:02.014561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c50da-b0c1-46a9-b240-75ea72e59413 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:02.014836 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c50da-b0c1-46a9-b240-75ea72e59413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:02.042847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-c88c50da-b0c1-46a9-b240-75ea72e59413 None None] MFA Rules not processed for user `bb0b0b973a90468a93064e28bed91a48`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:02.044041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-c88c50da-b0c1-46a9-b240-75ea72e59413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:02.111760 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1412/2819] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:36:02.123840 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:02.125778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:02.183803 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eef3e4-49bd-49f3-a31a-d2e6d757ae94 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:02.187850 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea2b2c-b5a6-44ed-826f-2c1aa161575e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:02.188459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea2b2c-b5a6-44ed-826f-2c1aa161575e service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:02.188736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea2b2c-b5a6-44ed-826f-2c1aa161575e service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:02.189005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea2b2c-b5a6-44ed-826f-2c1aa161575e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:02.197921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea2b2c-b5a6-44ed-826f-2c1aa161575e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:02.199344 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea2b2c-b5a6-44ed-826f-2c1aa161575e service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:02.209902 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1408/2820] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:02.282831 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c50da-b0c1-46a9-b240-75ea72e59413 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:02.292353 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c50da-b0c1-46a9-b240-75ea72e59413 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:02.296247 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098060c2-fe19-4bf0-ba16-b464124bda6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:02.296847 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098060c2-fe19-4bf0-ba16-b464124bda6c service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:02.297122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098060c2-fe19-4bf0-ba16-b464124bda6c service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:02.297390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098060c2-fe19-4bf0-ba16-b464124bda6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:02.352920 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098060c2-fe19-4bf0-ba16-b464124bda6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:02.354390 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098060c2-fe19-4bf0-ba16-b464124bda6c service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:02.363575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1413/2821] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:02.627093 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea2b2c-b5a6-44ed-826f-2c1aa161575e service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:02.689338 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea2b2c-b5a6-44ed-826f-2c1aa161575e service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:02.693817 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:02.694448 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:02.694736 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:02.694993 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] PATH_INFO: `/v3/users/bd446fea256f4d8eb7c199b0d9e8acfb` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:02.696144 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd446fea256f4d8eb7c199b0d9e8acfb)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:02.698108 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:02.731933 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:02.746006 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd446fea256f4d8eb7c199b0d9e8acfb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:02.753568 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd446fea256f4d8eb7c199b0d9e8acfb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:02.754181 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd446fea256f4d8eb7c199b0d9e8acfb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:02.766945 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd446fea256f4d8eb7c199b0d9e8acfb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:02.767575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd446fea256f4d8eb7c199b0d9e8acfb'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:02.779304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1409/2822] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:36:02 2022] DELETE /identity/v3/users/bd446fea256f4d8eb7c199b0d9e8acfb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:02.783934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b99c-f4fd-4265-a73c-4c5067c94f82 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:02.784225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b99c-f4fd-4265-a73c-4c5067c94f82 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:02.784500 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b99c-f4fd-4265-a73c-4c5067c94f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:02.804240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-2031b99c-f4fd-4265-a73c-4c5067c94f82 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:02.805474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2031b99c-f4fd-4265-a73c-4c5067c94f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:02.863145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1414/2823] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:02.872951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:02.875395 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:02.932349 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daf3f0-3ab6-4f82-bfd9-dcc9bce09a26 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:02.936096 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7251-f9ec-4842-8293-4ae1a21c5f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:02.937713 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7251-f9ec-4842-8293-4ae1a21c5f95 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:02.937994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7251-f9ec-4842-8293-4ae1a21c5f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:02.938266 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7251-f9ec-4842-8293-4ae1a21c5f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:02.947371 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f7251-f9ec-4842-8293-4ae1a21c5f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:02.948739 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f7251-f9ec-4842-8293-4ae1a21c5f95 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:02.958844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1410/2824] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:03.229934 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031b99c-f4fd-4265-a73c-4c5067c94f82 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:03.231425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-2031b99c-f4fd-4265-a73c-4c5067c94f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:03.239035 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:03.245344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031b99c-f4fd-4265-a73c-4c5067c94f82 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:03.250785 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:03.251436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:03.251717 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:03.251993 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] PATH_INFO: `/v3/projects/457b91686be64992b991199419ef15aa` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:03.253157 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=457b91686be64992b991199419ef15aa)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:03.255105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:03.301952 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457b91686be64992b991199419ef15aa', 'request_id': 'req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.307477 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457b91686be64992b991199419ef15aa', 'request_id': 'req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.315764 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457b91686be64992b991199419ef15aa', 'request_id': 'req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.326563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] The token cache is being invalidate because project 457b91686be64992b991199419ef15aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:36:03.327837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.328465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.330074 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1415/2825] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:36:03 2022] DELETE /identity/v3/projects/457b91686be64992b991199419ef15aa => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:03.334308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7251-f9ec-4842-8293-4ae1a21c5f95 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:03.344071 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7251-f9ec-4842-8293-4ae1a21c5f95 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:03.399239 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:03.399927 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:03.400231 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:03.400497 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] PATH_INFO: `/v3/users/bb0b0b973a90468a93064e28bed91a48` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:03.401683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0b0b973a90468a93064e28bed91a48)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:03.403615 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:03.449749 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0b0b973a90468a93064e28bed91a48'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.457865 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0b0b973a90468a93064e28bed91a48'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.458543 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0b0b973a90468a93064e28bed91a48'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.459659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:03.481077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0b0b973a90468a93064e28bed91a48'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.481978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-1db50005-e1b1-440a-ba97-8b6b6ddf040c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0b0b973a90468a93064e28bed91a48'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.495891 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1411/2826] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:36:03 2022] DELETE /identity/v3/users/bb0b0b973a90468a93064e28bed91a48 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:03.774570 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:03.775971 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:03.789325 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bd7fd-3073-43c7-9307-7a0d09ab83f1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:03.849628 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:03.850271 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:03.850547 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:03.850816 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] PATH_INFO: `/v3/projects/135a3a7067314226a38976efa00873b5` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:03.851956 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=135a3a7067314226a38976efa00873b5)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:03.853886 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:03.899422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135a3a7067314226a38976efa00873b5', 'request_id': 'req-99729221-df84-4bd7-8eef-f14c146b3726'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.905308 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135a3a7067314226a38976efa00873b5', 'request_id': 'req-99729221-df84-4bd7-8eef-f14c146b3726'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.912480 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135a3a7067314226a38976efa00873b5', 'request_id': 'req-99729221-df84-4bd7-8eef-f14c146b3726'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.924456 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] The token cache is being invalidate because project 135a3a7067314226a38976efa00873b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:36:03.924783 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.925469 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-99729221-df84-4bd7-8eef-f14c146b3726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:03.927067 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1416/2827] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:36:03 2022] DELETE /identity/v3/projects/135a3a7067314226a38976efa00873b5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:03.932013 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4e26-a879-4248-b84b-0f46d9c5dc84 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:03.932503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4e26-a879-4248-b84b-0f46d9c5dc84 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:03.932610 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4e26-a879-4248-b84b-0f46d9c5dc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:03.951772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-e6cb4e26-a879-4248-b84b-0f46d9c5dc84 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:03.952968 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-e6cb4e26-a879-4248-b84b-0f46d9c5dc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:04.011575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1412/2828] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:04.020707 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301d3e-a56d-4f69-8bb0-3d0518033d83 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.020914 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301d3e-a56d-4f69-8bb0-3d0518033d83 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.021313 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301d3e-a56d-4f69-8bb0-3d0518033d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.041341 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-05301d3e-a56d-4f69-8bb0-3d0518033d83 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:04.042607 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-05301d3e-a56d-4f69-8bb0-3d0518033d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:04.050521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1417/2829] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:04.056226 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb4e26-a879-4248-b84b-0f46d9c5dc84 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.066596 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb4e26-a879-4248-b84b-0f46d9c5dc84 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.071150 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.071762 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.072046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.072321 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.073041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:36:04.073348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.074986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.087778 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1413/2830] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:36:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:36:04.092254 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05301d3e-a56d-4f69-8bb0-3d0518033d83 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.101778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05301d3e-a56d-4f69-8bb0-3d0518033d83 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.106359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda722a-cb41-4370-ad30-5e9b8e690950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.106968 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda722a-cb41-4370-ad30-5e9b8e690950 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.107240 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda722a-cb41-4370-ad30-5e9b8e690950 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.107562 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda722a-cb41-4370-ad30-5e9b8e690950 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.108268 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda722a-cb41-4370-ad30-5e9b8e690950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.110297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda722a-cb41-4370-ad30-5e9b8e690950 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.115732 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:04.141645 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1418/2831] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/projects => generated 384 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:04.145898 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.156938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2fa4a-88c5-4d7a-9bf6-4dec8aaa9bb6 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.217664 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb21251-6058-45ff-ad41-cfa0a6f6cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.218283 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21251-6058-45ff-ad41-cfa0a6f6cc6a admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.218566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21251-6058-45ff-ad41-cfa0a6f6cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.218836 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21251-6058-45ff-ad41-cfa0a6f6cc6a admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.219504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb21251-6058-45ff-ad41-cfa0a6f6cc6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.221504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb21251-6058-45ff-ad41-cfa0a6f6cc6a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.243046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1414/2832] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:04.247281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda722a-cb41-4370-ad30-5e9b8e690950 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.258287 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda722a-cb41-4370-ad30-5e9b8e690950 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.262965 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a504751-acc5-4aa2-9303-75f719649d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.263571 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504751-acc5-4aa2-9303-75f719649d46 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.263851 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504751-acc5-4aa2-9303-75f719649d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.264127 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504751-acc5-4aa2-9303-75f719649d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.264852 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a504751-acc5-4aa2-9303-75f719649d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.266479 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a504751-acc5-4aa2-9303-75f719649d46 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.276233 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1419/2833] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:04 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:04.280037 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb21251-6058-45ff-ad41-cfa0a6f6cc6a admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.289720 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb21251-6058-45ff-ad41-cfa0a6f6cc6a admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.294260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e0336-04f9-4843-a7be-794859e1a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.294854 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e0336-04f9-4843-a7be-794859e1a9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.295136 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e0336-04f9-4843-a7be-794859e1a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.295956 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e0336-04f9-4843-a7be-794859e1a9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.296649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e0336-04f9-4843-a7be-794859e1a9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.298289 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e0336-04f9-4843-a7be-794859e1a9b9 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.309441 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1415/2834] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:04 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:04.314197 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a504751-acc5-4aa2-9303-75f719649d46 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.324083 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a504751-acc5-4aa2-9303-75f719649d46 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.329277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b927-fa5e-47b3-bf4b-135c77c22d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.329958 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b927-fa5e-47b3-bf4b-135c77c22d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.330239 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b927-fa5e-47b3-bf4b-135c77c22d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.330513 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b927-fa5e-47b3-bf4b-135c77c22d16 admin admin] PATH_INFO: `/v3/projects/ff16b3b361e945e782569ce37c15ef39/users/deae8e8733cf44188548e928d678e9c5/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.345372 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771b927-fa5e-47b3-bf4b-135c77c22d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff16b3b361e945e782569ce37c15ef39, user_id=deae8e8733cf44188548e928d678e9c5, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.348034 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771b927-fa5e-47b3-bf4b-135c77c22d16 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.361047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1420/2835] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:04 2022] PUT /identity/v3/projects/ff16b3b361e945e782569ce37c15ef39/users/deae8e8733cf44188548e928d678e9c5/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:04.370981 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62ad5-a4b4-4c2e-97f1-f942b9940eb5 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.371263 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62ad5-a4b4-4c2e-97f1-f942b9940eb5 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.371533 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62ad5-a4b4-4c2e-97f1-f942b9940eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.398980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-a5b62ad5-a4b4-4c2e-97f1-f942b9940eb5 None None] MFA Rules not processed for user `deae8e8733cf44188548e928d678e9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:04.400202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a5b62ad5-a4b4-4c2e-97f1-f942b9940eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:04.425730 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:04.465300 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1416/2836] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:04.470493 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b927-fa5e-47b3-bf4b-135c77c22d16 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.531721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b927-fa5e-47b3-bf4b-135c77c22d16 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.536186 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feecf8-a780-416f-b450-f55b4cb81e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.537441 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feecf8-a780-416f-b450-f55b4cb81e4a admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.537727 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feecf8-a780-416f-b450-f55b4cb81e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.537994 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feecf8-a780-416f-b450-f55b4cb81e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.538646 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feecf8-a780-416f-b450-f55b4cb81e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.540549 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feecf8-a780-416f-b450-f55b4cb81e4a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.566128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1421/2837] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/projects => generated 384 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:36:04.570173 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b62ad5-a4b4-4c2e-97f1-f942b9940eb5 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.581876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b62ad5-a4b4-4c2e-97f1-f942b9940eb5 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.638468 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afe19-030c-4735-a026-02d6f6aa62e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.639083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afe19-030c-4735-a026-02d6f6aa62e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.640424 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afe19-030c-4735-a026-02d6f6aa62e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.640735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afe19-030c-4735-a026-02d6f6aa62e5 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.641393 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2afe19-030c-4735-a026-02d6f6aa62e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.643345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2afe19-030c-4735-a026-02d6f6aa62e5 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.664101 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1417/2838] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:04.668075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feecf8-a780-416f-b450-f55b4cb81e4a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.678082 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feecf8-a780-416f-b450-f55b4cb81e4a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.682678 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e86c14-b895-49dd-95fd-5b4e197be431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.683278 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e86c14-b895-49dd-95fd-5b4e197be431 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.683559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e86c14-b895-49dd-95fd-5b4e197be431 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.683823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e86c14-b895-49dd-95fd-5b4e197be431 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.684501 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e86c14-b895-49dd-95fd-5b4e197be431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.686648 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e86c14-b895-49dd-95fd-5b4e197be431 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.696916 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1422/2839] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:04 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:04.700766 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afe19-030c-4735-a026-02d6f6aa62e5 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.711658 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afe19-030c-4735-a026-02d6f6aa62e5 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.716217 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02240d81-900c-4454-8f42-fe6f520d23ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.716811 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240d81-900c-4454-8f42-fe6f520d23ac admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.717088 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240d81-900c-4454-8f42-fe6f520d23ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.717362 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240d81-900c-4454-8f42-fe6f520d23ac admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.718111 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02240d81-900c-4454-8f42-fe6f520d23ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.719688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02240d81-900c-4454-8f42-fe6f520d23ac admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.729495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1418/2840] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:04 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:04.733588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e86c14-b895-49dd-95fd-5b4e197be431 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.743363 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e86c14-b895-49dd-95fd-5b4e197be431 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.747875 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b83893-18af-4023-80a0-f606df99e16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.749007 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b83893-18af-4023-80a0-f606df99e16a admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.749583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b83893-18af-4023-80a0-f606df99e16a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.749583 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b83893-18af-4023-80a0-f606df99e16a admin admin] PATH_INFO: `/v3/projects/057a648e545b4b3dbc9a8cf1c233a6f7/users/2f082a5aee034aedac8c091fa4366b1e/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.763858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b83893-18af-4023-80a0-f606df99e16a admin admin] RBAC: Authorizing `identity:create_grant(project_id=057a648e545b4b3dbc9a8cf1c233a6f7, user_id=2f082a5aee034aedac8c091fa4366b1e, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.767122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b83893-18af-4023-80a0-f606df99e16a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.779333 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1423/2841] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:04 2022] PUT /identity/v3/projects/057a648e545b4b3dbc9a8cf1c233a6f7/users/2f082a5aee034aedac8c091fa4366b1e/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:36:04.789347 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53263-2aaf-4524-a5dc-05cfe87021cf None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.789613 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53263-2aaf-4524-a5dc-05cfe87021cf None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.789873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53263-2aaf-4524-a5dc-05cfe87021cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.817581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-60b53263-2aaf-4524-a5dc-05cfe87021cf None None] MFA Rules not processed for user `2f082a5aee034aedac8c091fa4366b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:04.818841 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-60b53263-2aaf-4524-a5dc-05cfe87021cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:04.883629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1419/2842] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:04.888211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b83893-18af-4023-80a0-f606df99e16a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:04.951136 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b83893-18af-4023-80a0-f606df99e16a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:04.956885 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc40e4-a6d9-44cf-ae78-9c5711320589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:04.957518 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc40e4-a6d9-44cf-ae78-9c5711320589 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:04.957799 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc40e4-a6d9-44cf-ae78-9c5711320589 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:04.958066 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc40e4-a6d9-44cf-ae78-9c5711320589 admin admin] PATH_INFO: `/v3/projects` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:04.958713 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc40e4-a6d9-44cf-ae78-9c5711320589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:04.960580 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc40e4-a6d9-44cf-ae78-9c5711320589 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:04.965962 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:04.989474 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1424/2843] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/projects => generated 386 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:04.993662 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b53263-2aaf-4524-a5dc-05cfe87021cf None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:05.004544 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b53263-2aaf-4524-a5dc-05cfe87021cf None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:05.037293 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:05.062107 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabb46-a1c0-49d6-acef-9556a7d3a067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:05.062654 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabb46-a1c0-49d6-acef-9556a7d3a067 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:05.062931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabb46-a1c0-49d6-acef-9556a7d3a067 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:05.063202 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabb46-a1c0-49d6-acef-9556a7d3a067 admin admin] PATH_INFO: `/v3/users` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:05.063892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dabb46-a1c0-49d6-acef-9556a7d3a067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:05.065846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dabb46-a1c0-49d6-acef-9556a7d3a067 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:05.087046 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1420/2844] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:36:04 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:05.091033 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc40e4-a6d9-44cf-ae78-9c5711320589 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:05.101820 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc40e4-a6d9-44cf-ae78-9c5711320589 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:05.106936 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ffec2-8812-4a51-a38e-40387ac83871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:05.107546 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ffec2-8812-4a51-a38e-40387ac83871 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:05.107860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ffec2-8812-4a51-a38e-40387ac83871 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:05.108332 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ffec2-8812-4a51-a38e-40387ac83871 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:05.109203 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ffec2-8812-4a51-a38e-40387ac83871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:05.111015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ffec2-8812-4a51-a38e-40387ac83871 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:05.121256 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1425/2845] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:05 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:05.125659 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabb46-a1c0-49d6-acef-9556a7d3a067 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:05.136137 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabb46-a1c0-49d6-acef-9556a7d3a067 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:05.141391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fef61-1387-4be5-8902-681786ebc352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:05.142068 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fef61-1387-4be5-8902-681786ebc352 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:05.142348 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fef61-1387-4be5-8902-681786ebc352 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:05.142621 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fef61-1387-4be5-8902-681786ebc352 admin admin] PATH_INFO: `/v3/projects/d31f5722be194f10b38c911073ba5c91/users/d20c9213d0a24083a0cfa5251f999fdf/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:05.157660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fef61-1387-4be5-8902-681786ebc352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31f5722be194f10b38c911073ba5c91, user_id=d20c9213d0a24083a0cfa5251f999fdf, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:05.160358 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fef61-1387-4be5-8902-681786ebc352 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:05.173935 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1421/2846] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:05 2022] PUT /identity/v3/projects/d31f5722be194f10b38c911073ba5c91/users/d20c9213d0a24083a0cfa5251f999fdf/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:05.178858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ffec2-8812-4a51-a38e-40387ac83871 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:05.189444 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ffec2-8812-4a51-a38e-40387ac83871 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:05.245862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412bb3b-794b-4e51-9cd9-276cf52bf749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:05.246481 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412bb3b-794b-4e51-9cd9-276cf52bf749 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:05.246763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412bb3b-794b-4e51-9cd9-276cf52bf749 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:05.247037 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412bb3b-794b-4e51-9cd9-276cf52bf749 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:05.247726 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412bb3b-794b-4e51-9cd9-276cf52bf749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:05.249619 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412bb3b-794b-4e51-9cd9-276cf52bf749 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:05.259334 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1426/2847] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:05 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:05.263687 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fef61-1387-4be5-8902-681786ebc352 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:05.274503 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fef61-1387-4be5-8902-681786ebc352 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:05.279062 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:05.280352 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:05.280775 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:05.281042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] PATH_INFO: `/v3/projects/d31f5722be194f10b38c911073ba5c91/users/d20c9213d0a24083a0cfa5251f999fdf/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:05.283030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31f5722be194f10b38c911073ba5c91, user_id=d20c9213d0a24083a0cfa5251f999fdf, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:05.285679 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:05.295781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1422/2848] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:05 2022] PUT /identity/v3/projects/d31f5722be194f10b38c911073ba5c91/users/d20c9213d0a24083a0cfa5251f999fdf/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:05.307437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde3bd-ef4e-4f4c-ba29-ec17b7a08cc3 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:05.307725 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde3bd-ef4e-4f4c-ba29-ec17b7a08cc3 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:05.308002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde3bd-ef4e-4f4c-ba29-ec17b7a08cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:05.336944 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-93dde3bd-ef4e-4f4c-ba29-ec17b7a08cc3 None None] MFA Rules not processed for user `d20c9213d0a24083a0cfa5251f999fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:05.338212 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-93dde3bd-ef4e-4f4c-ba29-ec17b7a08cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:05.405986 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1427/2849] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:05 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:05.417129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:05.418892 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:05.475360 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491c1a-d467-489b-a3fa-f7620412e261 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:05.479113 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be497e-a069-4c89-8bda-dd6df4e7a819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:05.479721 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be497e-a069-4c89-8bda-dd6df4e7a819 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:05.480002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be497e-a069-4c89-8bda-dd6df4e7a819 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:05.480278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be497e-a069-4c89-8bda-dd6df4e7a819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:05.536495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be497e-a069-4c89-8bda-dd6df4e7a819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:05.537988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be497e-a069-4c89-8bda-dd6df4e7a819 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:05.546890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1423/2850] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:07.043843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dde3bd-ef4e-4f4c-ba29-ec17b7a08cc3 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:07.045322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-93dde3bd-ef4e-4f4c-ba29-ec17b7a08cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:07.057172 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dde3bd-ef4e-4f4c-ba29-ec17b7a08cc3 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:07.060719 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da77e15-929a-4831-80c3-484e97461dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:07.061870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da77e15-929a-4831-80c3-484e97461dab service cinder] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:07.062148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da77e15-929a-4831-80c3-484e97461dab service cinder] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:07.062419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da77e15-929a-4831-80c3-484e97461dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:07.116843 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:07.119556 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da77e15-929a-4831-80c3-484e97461dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:07.121524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da77e15-929a-4831-80c3-484e97461dab service cinder] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:07.132808 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1428/2851] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:07.666540 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be497e-a069-4c89-8bda-dd6df4e7a819 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:07.675716 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be497e-a069-4c89-8bda-dd6df4e7a819 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:07.679318 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a1283-9900-4c08-99ec-8c045891c4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:07.679920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a1283-9900-4c08-99ec-8c045891c4ff service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:07.680193 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a1283-9900-4c08-99ec-8c045891c4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:07.680460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a1283-9900-4c08-99ec-8c045891c4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:07.689342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a1283-9900-4c08-99ec-8c045891c4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:07.690796 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a1283-9900-4c08-99ec-8c045891c4ff service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:07.700031 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1424/2852] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:07 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:11.549202 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da77e15-929a-4831-80c3-484e97461dab service cinder] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:11.610473 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da77e15-929a-4831-80c3-484e97461dab service cinder] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:11.615190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:11.615839 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:11.616118 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:11.616425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] PATH_INFO: `/v3/users/deae8e8733cf44188548e928d678e9c5` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:11.617749 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deae8e8733cf44188548e928d678e9c5)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:11.619983 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:11.668475 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deae8e8733cf44188548e928d678e9c5'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:11.675794 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deae8e8733cf44188548e928d678e9c5'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:11.683032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deae8e8733cf44188548e928d678e9c5'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:11.683644 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deae8e8733cf44188548e928d678e9c5'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:11.696128 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deae8e8733cf44188548e928d678e9c5'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:11.703029 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1429/2853] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:36:11 2022] DELETE /identity/v3/users/deae8e8733cf44188548e928d678e9c5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:11.707755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0268c-7f06-403f-9c46-f5f6051f77e7 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:11.708259 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0268c-7f06-403f-9c46-f5f6051f77e7 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:11.708374 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0268c-7f06-403f-9c46-f5f6051f77e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:11.727311 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8da0268c-7f06-403f-9c46-f5f6051f77e7 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:11.728547 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8da0268c-7f06-403f-9c46-f5f6051f77e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:11.753340 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:11.789719 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1425/2854] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:11.799870 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:11.801328 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:11.858413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe581827-d8bb-4546-83f6-a7841e5c63f3 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:11.862135 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60185c9-cfda-4ad4-a0c9-9dc8b0327517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:11.862743 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60185c9-cfda-4ad4-a0c9-9dc8b0327517 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:11.863023 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60185c9-cfda-4ad4-a0c9-9dc8b0327517 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:11.863296 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60185c9-cfda-4ad4-a0c9-9dc8b0327517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:11.872314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60185c9-cfda-4ad4-a0c9-9dc8b0327517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:11.874382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60185c9-cfda-4ad4-a0c9-9dc8b0327517 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:11.883873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1430/2855] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:12.155978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0268c-7f06-403f-9c46-f5f6051f77e7 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:12.157512 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8da0268c-7f06-403f-9c46-f5f6051f77e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:12.170628 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0268c-7f06-403f-9c46-f5f6051f77e7 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:12.175317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:12.175967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:12.176248 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:12.176524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] PATH_INFO: `/v3/projects/ff16b3b361e945e782569ce37c15ef39` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:12.177729 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff16b3b361e945e782569ce37c15ef39)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:12.179999 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:12.226921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff16b3b361e945e782569ce37c15ef39', 'request_id': 'req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.234522 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff16b3b361e945e782569ce37c15ef39', 'request_id': 'req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.240851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff16b3b361e945e782569ce37c15ef39', 'request_id': 'req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.252020 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] The token cache is being invalidate because project ff16b3b361e945e782569ce37c15ef39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:36:12.252306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.252951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.254495 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1426/2856] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:36:12 2022] DELETE /identity/v3/projects/ff16b3b361e945e782569ce37c15ef39 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:12.258459 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60185c9-cfda-4ad4-a0c9-9dc8b0327517 service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:12.268125 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60185c9-cfda-4ad4-a0c9-9dc8b0327517 service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:12.280810 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:12.325424 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:12.326269 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:12.326557 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:12.326946 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] PATH_INFO: `/v3/users/2f082a5aee034aedac8c091fa4366b1e` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:12.328130 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f082a5aee034aedac8c091fa4366b1e)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:12.330088 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:12.376436 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f082a5aee034aedac8c091fa4366b1e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.383047 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f082a5aee034aedac8c091fa4366b1e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.390017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f082a5aee034aedac8c091fa4366b1e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.390622 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f082a5aee034aedac8c091fa4366b1e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.402277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f082a5aee034aedac8c091fa4366b1e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.409000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1431/2857] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:36:12 2022] DELETE /identity/v3/users/2f082a5aee034aedac8c091fa4366b1e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:12.683619 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:12.685044 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:12.698018 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb957-8c2f-4387-824d-6374e3bd7e5f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:12.757710 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:12.758410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:12.758708 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:12.758986 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] PATH_INFO: `/v3/projects/057a648e545b4b3dbc9a8cf1c233a6f7` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:12.760137 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057a648e545b4b3dbc9a8cf1c233a6f7)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:12.762091 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:12.808042 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057a648e545b4b3dbc9a8cf1c233a6f7', 'request_id': 'req-dea07a3e-43cb-4668-a5de-1818abe4c987'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.811781 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:12.815436 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057a648e545b4b3dbc9a8cf1c233a6f7', 'request_id': 'req-dea07a3e-43cb-4668-a5de-1818abe4c987'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.822048 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057a648e545b4b3dbc9a8cf1c233a6f7', 'request_id': 'req-dea07a3e-43cb-4668-a5de-1818abe4c987'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.833484 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] The token cache is being invalidate because project 057a648e545b4b3dbc9a8cf1c233a6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:36:12.833767 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.834390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.835867 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1427/2858] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:36:12 2022] DELETE /identity/v3/projects/057a648e545b4b3dbc9a8cf1c233a6f7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:12.839823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:12.841273 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:12.862980 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867bf97-dd35-4b0a-9bc3-8f0b0a472a29 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:12.919716 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:12.920797 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:12.921077 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:12.921347 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] PATH_INFO: `/v3/users/d20c9213d0a24083a0cfa5251f999fdf` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:12.922560 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20c9213d0a24083a0cfa5251f999fdf)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:12.924476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:12.964769 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:12.972152 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20c9213d0a24083a0cfa5251f999fdf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.979860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20c9213d0a24083a0cfa5251f999fdf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.987521 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20c9213d0a24083a0cfa5251f999fdf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:12.988139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20c9213d0a24083a0cfa5251f999fdf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:13.001075 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-b76c27c7-ad16-40cd-b23b-e1a0f2b5b908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20c9213d0a24083a0cfa5251f999fdf'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:13.006846 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1432/2859] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:36:12 2022] DELETE /identity/v3/users/d20c9213d0a24083a0cfa5251f999fdf => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:13.276987 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:13.278537 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:13.292831 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea07a3e-43cb-4668-a5de-1818abe4c987 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:13.352649 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:13.353212 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.353624 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.353899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] PATH_INFO: `/v3/projects/d31f5722be194f10b38c911073ba5c91` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.355041 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d31f5722be194f10b38c911073ba5c91)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:13.356951 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:13.403342 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd31f5722be194f10b38c911073ba5c91', 'request_id': 'req-a713d473-a833-4efa-8f10-aaee7e67377a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:13.410474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd31f5722be194f10b38c911073ba5c91', 'request_id': 'req-a713d473-a833-4efa-8f10-aaee7e67377a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:13.417967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd31f5722be194f10b38c911073ba5c91', 'request_id': 'req-a713d473-a833-4efa-8f10-aaee7e67377a'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:13.430967 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] The token cache is being invalidate because project d31f5722be194f10b38c911073ba5c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:36:13.431258 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:13.431914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a713d473-a833-4efa-8f10-aaee7e67377a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:36:13.433479 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1428/2860] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:36:13 2022] DELETE /identity/v3/projects/d31f5722be194f10b38c911073ba5c91 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:13.439010 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa291-2c30-4b68-8523-4c523a47eb83 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.439297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa291-2c30-4b68-8523-4c523a47eb83 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.439567 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa291-2c30-4b68-8523-4c523a47eb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.459046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-270fa291-2c30-4b68-8523-4c523a47eb83 None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:13.460252 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-270fa291-2c30-4b68-8523-4c523a47eb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:13.518613 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1433/2861] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 22:36:13.526690 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cce59e-701c-4b26-a65a-da7f8624f99b None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.527131 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cce59e-701c-4b26-a65a-da7f8624f99b None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.527418 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cce59e-701c-4b26-a65a-da7f8624f99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.547411 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-55cce59e-701c-4b26-a65a-da7f8624f99b None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:13.548622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-55cce59e-701c-4b26-a65a-da7f8624f99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:13.556557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1429/2862] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:13.561364 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fa291-2c30-4b68-8523-4c523a47eb83 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:13.571733 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fa291-2c30-4b68-8523-4c523a47eb83 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:13.576297 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:13.576901 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.577184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.577559 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] PATH_INFO: `/v3/domains` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.578251 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 22:36:13.578573 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:13.580419 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:13.592940 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1434/2863] 10.222.0.109 () {58 vars in 1158 bytes} [Thu Aug 4 22:36:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 22:36:13.597190 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cce59e-701c-4b26-a65a-da7f8624f99b None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:13.606330 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cce59e-701c-4b26-a65a-da7f8624f99b None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:13.614273 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e7c53-9fba-4d13-a007-152620610415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:13.614879 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e7c53-9fba-4d13-a007-152620610415 admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.615159 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e7c53-9fba-4d13-a007-152620610415 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.615430 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e7c53-9fba-4d13-a007-152620610415 admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.616099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e7c53-9fba-4d13-a007-152620610415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:13.618129 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e7c53-9fba-4d13-a007-152620610415 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:13.645873 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1430/2864] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:36:13 2022] POST /identity/v3/projects => generated 384 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:13.650119 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:13.660422 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845be3-e456-4564-8ed4-336e7432f0cb admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:13.716266 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe9242-4843-4684-acc7-d17037e969b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:13.717041 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe9242-4843-4684-acc7-d17037e969b1 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.717322 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe9242-4843-4684-acc7-d17037e969b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.717621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe9242-4843-4684-acc7-d17037e969b1 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.718264 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe9242-4843-4684-acc7-d17037e969b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:13.720148 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe9242-4843-4684-acc7-d17037e969b1 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:13.740778 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1435/2865] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:36:13 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:13.744950 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e7c53-9fba-4d13-a007-152620610415 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:13.755923 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e7c53-9fba-4d13-a007-152620610415 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:13.760597 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cef71-710c-451c-ac6e-cf09bbbcf1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:13.761207 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cef71-710c-451c-ac6e-cf09bbbcf1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.761524 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cef71-710c-451c-ac6e-cf09bbbcf1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.761799 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cef71-710c-451c-ac6e-cf09bbbcf1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.762496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cef71-710c-451c-ac6e-cf09bbbcf1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:13.764725 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cef71-710c-451c-ac6e-cf09bbbcf1b0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:13.774493 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1431/2866] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:13 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:13.778277 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe9242-4843-4684-acc7-d17037e969b1 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:13.788435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe9242-4843-4684-acc7-d17037e969b1 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:13.793183 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463604c-4640-467e-85a9-61d40c50d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:13.793819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463604c-4640-467e-85a9-61d40c50d532 admin admin] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.794094 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463604c-4640-467e-85a9-61d40c50d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.794359 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463604c-4640-467e-85a9-61d40c50d532 admin admin] PATH_INFO: `/v3/roles` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.795155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463604c-4640-467e-85a9-61d40c50d532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:13.796659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463604c-4640-467e-85a9-61d40c50d532 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:13.806429 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1436/2867] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:13 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:13.810812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cef71-710c-451c-ac6e-cf09bbbcf1b0 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:13.820751 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cef71-710c-451c-ac6e-cf09bbbcf1b0 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:13.825496 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:13.826773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.827054 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.827356 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] PATH_INFO: `/v3/projects/34dc6542be764622b304ee5f895041a2/users/285c9f1fe752403fb7374b490391384e/roles/bad0ac6580d5470fb6dd23a137576b9c` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.842984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34dc6542be764622b304ee5f895041a2, user_id=285c9f1fe752403fb7374b490391384e, role_id=bad0ac6580d5470fb6dd23a137576b9c)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:13.845646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:13.858155 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1432/2868] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:13 2022] PUT /identity/v3/projects/34dc6542be764622b304ee5f895041a2/users/285c9f1fe752403fb7374b490391384e/roles/bad0ac6580d5470fb6dd23a137576b9c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:36:13.863100 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a5a8-3be2-4eba-8375-0223966a785a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:13.863565 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a5a8-3be2-4eba-8375-0223966a785a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:13.863691 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a5a8-3be2-4eba-8375-0223966a785a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:13.883391 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-fa26a5a8-3be2-4eba-8375-0223966a785a None None] MFA Rules not processed for user `f59916fe4d6d4a6aac1fc1ff790ac163`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:13.884565 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-fa26a5a8-3be2-4eba-8375-0223966a785a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:13.901453 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:13.944618 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1437/2869] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:13.955081 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:13.957007 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:13.967629 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:14.016005 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46879039-53cc-4755-9e57-225ac3e72af0 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:14.019691 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1399ba-b1d1-405b-b07d-c7297809d714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:14.020309 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1399ba-b1d1-405b-b07d-c7297809d714 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:14.020598 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1399ba-b1d1-405b-b07d-c7297809d714 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:14.020871 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1399ba-b1d1-405b-b07d-c7297809d714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:14.030336 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1399ba-b1d1-405b-b07d-c7297809d714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:14.031802 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1399ba-b1d1-405b-b07d-c7297809d714 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:14.041682 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1433/2870] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:18.026218 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63744e4-a35c-41ae-a291-f519495ba77a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:18.027051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63744e4-a35c-41ae-a291-f519495ba77a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:18.027349 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63744e4-a35c-41ae-a291-f519495ba77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:18.062216 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-e63744e4-a35c-41ae-a291-f519495ba77a None None] MFA Rules not processed for user `285c9f1fe752403fb7374b490391384e`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:18.063638 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-e63744e4-a35c-41ae-a291-f519495ba77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:18.141131 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1438/2871] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:18.145511 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1399ba-b1d1-405b-b07d-c7297809d714 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:18.155936 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1399ba-b1d1-405b-b07d-c7297809d714 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:18.160912 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab3d94-1a29-45bf-bb4b-7c5ad0b8a07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:18.161573 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab3d94-1a29-45bf-bb4b-7c5ad0b8a07e admin admin] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:18.161884 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab3d94-1a29-45bf-bb4b-7c5ad0b8a07e admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:18.162164 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab3d94-1a29-45bf-bb4b-7c5ad0b8a07e admin admin] PATH_INFO: `/v3/projects` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:18.162833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab3d94-1a29-45bf-bb4b-7c5ad0b8a07e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:18.165765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab3d94-1a29-45bf-bb4b-7c5ad0b8a07e admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:18.195444 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1434/2872] 10.222.0.109 () {60 vars in 1158 bytes} [Thu Aug 4 22:36:18 2022] POST /identity/v3/projects => generated 386 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 22:36:18.199586 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63744e4-a35c-41ae-a291-f519495ba77a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:18.210873 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63744e4-a35c-41ae-a291-f519495ba77a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:18.272659 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef8a88-781b-4659-b773-e4583081bf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:18.273289 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef8a88-781b-4659-b773-e4583081bf19 admin admin] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:18.273606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef8a88-781b-4659-b773-e4583081bf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:18.273879 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef8a88-781b-4659-b773-e4583081bf19 admin admin] PATH_INFO: `/v3/users` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:18.274561 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef8a88-781b-4659-b773-e4583081bf19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:18.276476 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef8a88-781b-4659-b773-e4583081bf19 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:18.305115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1439/2873] 10.222.0.109 () {60 vars in 1149 bytes} [Thu Aug 4 22:36:18 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 22:36:18.306648 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab3d94-1a29-45bf-bb4b-7c5ad0b8a07e admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:18.318148 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab3d94-1a29-45bf-bb4b-7c5ad0b8a07e admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:18.322888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c63a2-639c-4ef0-b2d1-1d8801f3d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:18.323530 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c63a2-639c-4ef0-b2d1-1d8801f3d7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:18.323812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c63a2-639c-4ef0-b2d1-1d8801f3d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:18.324085 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c63a2-639c-4ef0-b2d1-1d8801f3d7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:18.325246 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c63a2-639c-4ef0-b2d1-1d8801f3d7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:18.328253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c63a2-639c-4ef0-b2d1-1d8801f3d7a4 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:18.344145 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1435/2874] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:18 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:18.348437 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef8a88-781b-4659-b773-e4583081bf19 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:18.359988 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef8a88-781b-4659-b773-e4583081bf19 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:18.364849 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9a69-acd2-4b46-afd2-f5ec4efcd57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:18.365594 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9a69-acd2-4b46-afd2-f5ec4efcd57d admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:18.365878 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9a69-acd2-4b46-afd2-f5ec4efcd57d admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:18.366155 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9a69-acd2-4b46-afd2-f5ec4efcd57d admin admin] PATH_INFO: `/v3/projects/94fa04d4f22d45c5a7a7e080fb2645f9/users/cf751aab9be7453d85fe7f9a4bde2edd/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:18.382835 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be9a69-acd2-4b46-afd2-f5ec4efcd57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fa04d4f22d45c5a7a7e080fb2645f9, user_id=cf751aab9be7453d85fe7f9a4bde2edd, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:18.385665 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be9a69-acd2-4b46-afd2-f5ec4efcd57d admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:18.402105 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1440/2875] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:18 2022] PUT /identity/v3/projects/94fa04d4f22d45c5a7a7e080fb2645f9/users/cf751aab9be7453d85fe7f9a4bde2edd/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:36:18.408077 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c63a2-639c-4ef0-b2d1-1d8801f3d7a4 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:18.418622 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c63a2-639c-4ef0-b2d1-1d8801f3d7a4 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:18.488606 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13602115-c6b2-452c-b7e0-28394ece55d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:18.489260 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13602115-c6b2-452c-b7e0-28394ece55d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:18.489566 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13602115-c6b2-452c-b7e0-28394ece55d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:18.489833 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13602115-c6b2-452c-b7e0-28394ece55d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:18.490656 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13602115-c6b2-452c-b7e0-28394ece55d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:18.492253 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13602115-c6b2-452c-b7e0-28394ece55d3 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:18.502920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1436/2876] 10.222.0.109 () {58 vars in 1127 bytes} [Thu Aug 4 22:36:18 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 22:36:18.507008 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9a69-acd2-4b46-afd2-f5ec4efcd57d admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:18.517146 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9a69-acd2-4b46-afd2-f5ec4efcd57d admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:18.521699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:18.522344 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:18.522615 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:18.522880 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] PATH_INFO: `/v3/projects/94fa04d4f22d45c5a7a7e080fb2645f9/users/cf751aab9be7453d85fe7f9a4bde2edd/roles/68c4b28369a0474b83e1602e8e11c41c` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:18.524787 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fa04d4f22d45c5a7a7e080fb2645f9, user_id=cf751aab9be7453d85fe7f9a4bde2edd, role_id=68c4b28369a0474b83e1602e8e11c41c)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:18.527421 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:18.538577 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1441/2877] 10.222.0.109 () {60 vars in 1488 bytes} [Thu Aug 4 22:36:18 2022] PUT /identity/v3/projects/94fa04d4f22d45c5a7a7e080fb2645f9/users/cf751aab9be7453d85fe7f9a4bde2edd/roles/68c4b28369a0474b83e1602e8e11c41c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:36:23.047317 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb402f0-584f-4d03-8bf5-36c29cecc90a None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:23.047851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb402f0-584f-4d03-8bf5-36c29cecc90a None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:23.047851 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb402f0-584f-4d03-8bf5-36c29cecc90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:23.080264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-8cb402f0-584f-4d03-8bf5-36c29cecc90a None None] MFA Rules not processed for user `cf751aab9be7453d85fe7f9a4bde2edd`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:23.082052 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8cb402f0-584f-4d03-8bf5-36c29cecc90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:23.157966 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:23.162474 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1437/2878] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:23 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:23.176449 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:23.178257 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:23.185276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:23.264771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150c8ef-13c4-4296-b7c8-f1e7253866f4 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:23.268590 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33183e28de8f487f82f92f0151e7c73c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:23.269220 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:23.269536 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:23.269814 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:23.335642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:23.337184 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:23.349126 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1442/2879] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:23 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:23.494513 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25da82-6e52-4b80-bfba-1a867790486c None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:23.494795 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25da82-6e52-4b80-bfba-1a867790486c None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:23.495065 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25da82-6e52-4b80-bfba-1a867790486c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:23.515626 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-0c25da82-6e52-4b80-bfba-1a867790486c None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:23.517142 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-0c25da82-6e52-4b80-bfba-1a867790486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:23.584233 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1438/2880] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:36:23 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:23.635662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:23.637241 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:23.727215 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38399886-fbe2-4978-9233-d144a8bca10f service glance] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:23.732824 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2272f5-616b-4b36-b7a0-5c5943e33316 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:23.733465 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2272f5-616b-4b36-b7a0-5c5943e33316 service swift] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:23.733746 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2272f5-616b-4b36-b7a0-5c5943e33316 service swift] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:23.734011 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2272f5-616b-4b36-b7a0-5c5943e33316 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:23.743070 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2272f5-616b-4b36-b7a0-5c5943e33316 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:23.744676 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2272f5-616b-4b36-b7a0-5c5943e33316 service swift] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:23.754204 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1443/2881] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:36:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:36:29.057557 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25da82-6e52-4b80-bfba-1a867790486c None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:29.145296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25da82-6e52-4b80-bfba-1a867790486c None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:29.150206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:29.150834 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:29.151115 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:29.151391 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:29.154419 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:29.166396 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:29.167978 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:29.185914 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1439/2882] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:29 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:33.473663 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d9d91-5776-463f-8b3b-2e72ecd53dc4 None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:33.474145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d9d91-5776-463f-8b3b-2e72ecd53dc4 None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:33.474276 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d9d91-5776-463f-8b3b-2e72ecd53dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:33.499430 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-762d9d91-5776-463f-8b3b-2e72ecd53dc4 None None] MFA Rules not processed for user `285c9f1fe752403fb7374b490391384e`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:33.500624 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-762d9d91-5776-463f-8b3b-2e72ecd53dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:33.507517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1444/2883] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:33.765308 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:33.774106 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838da01-0fca-4ff8-89fb-219fa08c0cd4 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:33.778027 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1b381-7a63-4b24-a80d-c8d34c1db1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:33.778617 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1b381-7a63-4b24-a80d-c8d34c1db1fb service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:33.778896 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1b381-7a63-4b24-a80d-c8d34c1db1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:33.779169 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1b381-7a63-4b24-a80d-c8d34c1db1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:33.780920 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2aa1b381-7a63-4b24-a80d-c8d34c1db1fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:33.790567 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1b381-7a63-4b24-a80d-c8d34c1db1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:33.792000 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1b381-7a63-4b24-a80d-c8d34c1db1fb service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:33.800639 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1440/2884] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:33 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:34.132000 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d9d91-5776-463f-8b3b-2e72ecd53dc4 None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:34.133483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-762d9d91-5776-463f-8b3b-2e72ecd53dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:34.170249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=100 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:34.206307 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d9d91-5776-463f-8b3b-2e72ecd53dc4 None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:34.211211 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:34.211823 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:34.212095 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:34.212358 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:34.214081 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:34.225788 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:34.227345 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:34.238051 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1445/2885] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:34 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:36:35.577584 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26e18-244d-43bd-a961-0330cbfe42e9 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:35.578110 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26e18-244d-43bd-a961-0330cbfe42e9 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:35.578220 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d26e18-244d-43bd-a961-0330cbfe42e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:35.599700 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-19d26e18-244d-43bd-a961-0330cbfe42e9 None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:35.601024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-19d26e18-244d-43bd-a961-0330cbfe42e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:35.609392 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1441/2886] 10.222.0.109 () {58 vars in 1014 bytes} [Thu Aug 4 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:35.621856 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:35.634097 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea88a3-bf7b-418b-b812-5ac476f38f48 service placement] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:35.639255 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:35.639882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:35.640163 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:35.640434 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:35.648906 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:35.650524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:35.660755 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1446/2887] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:36:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 22:36:45.594826 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=60 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:45.656503 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=24 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:36:52.480359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb1a6-3a65-4622-be23-b3065742a948 None None] REQUEST_METHOD: `POST` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:52.480846 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb1a6-3a65-4622-be23-b3065742a948 None None] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:52.480962 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb1a6-3a65-4622-be23-b3065742a948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:52.499753 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.auth.core [None req-473cb1a6-3a65-4622-be23-b3065742a948 None None] MFA Rules not processed for user `43e543cfa3e1402e9694dcb095a26c00`. Rule list: `[]` (Enabled: `True`). {{(pid=91701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:36:52.500952 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-473cb1a6-3a65-4622-be23-b3065742a948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:52.554304 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1442/2888] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:36:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:36:53.648145 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:36:53.649699 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:53.731524 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb427b2a-63d8-4aee-bfa1-f3e72c1bfbdf service swift] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:36:53.737004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:36:53.737647 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:36:53.737928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:36:53.738201 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:36:53.739969 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:36:53.752588 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:36:53.754143 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:36:53.764780 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1447/2889] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:36:53 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:37:00.747812 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cb1a6-3a65-4622-be23-b3065742a948 None None] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:37:00.749886 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-473cb1a6-3a65-4622-be23-b3065742a948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:37:00.768717 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cb1a6-3a65-4622-be23-b3065742a948 None None] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:37:00.774355 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:37:00.774992 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:37:00.775980 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:37:00.776268 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:37:00.778746 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:37:00.794221 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:37:00.796359 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:37:00.808099 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1443/2890] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:37:00 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:37:03.760267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=34 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:37:06.677891 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:37:06.679312 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:37:06.691537 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16364a5-04b5-4ba7-9d9c-c30197fc9f40 service nova] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:37:06.695563 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bf36a-949f-4baa-a916-94a0e5930f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:37:06.696176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf36a-949f-4baa-a916-94a0e5930f24 service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:37:06.696450 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf36a-949f-4baa-a916-94a0e5930f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:37:06.697393 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf36a-949f-4baa-a916-94a0e5930f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:37:06.707664 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bf36a-949f-4baa-a916-94a0e5930f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:37:06.709234 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bf36a-949f-4baa-a916-94a0e5930f24 service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:37:06.722575 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1448/2891] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:37:06 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:37:10.803242 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=31 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:37:16.715792 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=6 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:39:11.458059 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:11.459466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:11.543039 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06333109-42d7-4e9b-99c8-91eb6986683c service nova] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:11.546708 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0422cd82114045a9fff7df29bc3906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:11.547312 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:11.547587 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:11.547855 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:11.549620 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:11.573969 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:11.575415 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:11.584913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1444/2892] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:39:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:39:21.580870 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=45 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:39:22.592520 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3aabf-03fc-43dd-91fc-0f669f81039d None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:22.593046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3aabf-03fc-43dd-91fc-0f669f81039d None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:22.593046 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3aabf-03fc-43dd-91fc-0f669f81039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:22.611748 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-0ae3aabf-03fc-43dd-91fc-0f669f81039d None None] MFA Rules not processed for user `43e543cfa3e1402e9694dcb095a26c00`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:39:22.612918 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0ae3aabf-03fc-43dd-91fc-0f669f81039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:22.619631 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1449/2893] 10.222.0.109 () {58 vars in 963 bytes} [Thu Aug 4 22:39:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:39:22.934166 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:22.954931 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2a021-a907-4e8f-8e97-56032ca45553 service cinder] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:22.958583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:22.959191 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:22.959470 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:22.959744 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:22.961504 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:22.985032 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:22.986642 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:22.995109 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1445/2894] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:39:22 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:39:23.911225 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3aabf-03fc-43dd-91fc-0f669f81039d None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:23.939858 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3aabf-03fc-43dd-91fc-0f669f81039d None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:23.944909 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0764709-db93-435f-9d33-41ec42322379 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:23.945569 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764709-db93-435f-9d33-41ec42322379 service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:23.945853 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764709-db93-435f-9d33-41ec42322379 service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:23.946122 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764709-db93-435f-9d33-41ec42322379 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:23.948214 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-d0764709-db93-435f-9d33-41ec42322379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:23.972696 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0764709-db93-435f-9d33-41ec42322379 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:23.974301 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0764709-db93-435f-9d33-41ec42322379 service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:23.984425 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1450/2895] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:39:23 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:39:26.765306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:26.823389 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854054d-0fb4-494d-b304-d31ec64120d3 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:26.833938 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41d9317e24714de1a310198bb35eb00b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:26.835004 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:26.835306 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:26.835575 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:26.838076 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:26.866792 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:26.868402 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:26.884881 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1446/2896] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:39:26 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:39:33.980071 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=21 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:39:35.738415 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708226e0-a75d-4108-937b-c9fb2542bd1a None None] REQUEST_METHOD: `POST` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:35.739017 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708226e0-a75d-4108-937b-c9fb2542bd1a None None] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:35.739190 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708226e0-a75d-4108-937b-c9fb2542bd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:35.760230 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.auth.core [None req-708226e0-a75d-4108-937b-c9fb2542bd1a None None] MFA Rules not processed for user `212a75e45f2b449398acbfb80338afb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 22:39:35.761609 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-708226e0-a75d-4108-937b-c9fb2542bd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:35.769621 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1451/2897] 10.222.0.109 () {58 vars in 972 bytes} [Thu Aug 4 22:39:35 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 22:39:35.812366 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:35.823890 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca69668-e557-4a81-924d-6bf32752b6de service nova] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:35.828876 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22724df4-c37e-4dc3-8ee9-4032d9420b40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848151a99fb74f6b8f8210cfc49afeff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:35.829502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22724df4-c37e-4dc3-8ee9-4032d9420b40 service swift] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:35.829805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22724df4-c37e-4dc3-8ee9-4032d9420b40 service swift] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:35.830074 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22724df4-c37e-4dc3-8ee9-4032d9420b40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:35.837913 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22724df4-c37e-4dc3-8ee9-4032d9420b40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:35.839477 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22724df4-c37e-4dc3-8ee9-4032d9420b40 service swift] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:35.848139 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1447/2898] 10.222.0.109 () {60 vars in 1389 bytes} [Thu Aug 4 22:39:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 22:39:39.582837 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708226e0-a75d-4108-937b-c9fb2542bd1a None None] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:39.584704 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-708226e0-a75d-4108-937b-c9fb2542bd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:39.602933 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708226e0-a75d-4108-937b-c9fb2542bd1a None None] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:39.610680 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:39.611435 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:39.611819 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:39.612195 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:39.614506 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:39.633721 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:39.635747 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:39.656921 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1452/2899] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:39:39 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:39:39.724888 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22724df4-c37e-4dc3-8ee9-4032d9420b40 service swift] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:39.735994 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22724df4-c37e-4dc3-8ee9-4032d9420b40 service swift] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:39.741805 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:39.742735 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:39.743015 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:39.743334 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:39.745581 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:39.833874 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:39.836677 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:39.845466 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1448/2900] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:39:39 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:39:43.326263 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:43.415842 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a26a-e3bf-4b33-80f1-1731fa6030f1 service placement] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:43.421683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:43.422517 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:43.422801 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:43.423443 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] PATH_INFO: `/v3/users/285c9f1fe752403fb7374b490391384e` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:43.424642 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=285c9f1fe752403fb7374b490391384e)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:43.426862 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:43.472888 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '285c9f1fe752403fb7374b490391384e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.479657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '285c9f1fe752403fb7374b490391384e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.487693 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '285c9f1fe752403fb7374b490391384e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.488319 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '285c9f1fe752403fb7374b490391384e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.500267 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '285c9f1fe752403fb7374b490391384e'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.506207 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1453/2901] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:39:43 2022] DELETE /identity/v3/users/285c9f1fe752403fb7374b490391384e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:39:43.684660 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:43.697683 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4d23b-be49-4dc4-96a9-43a4f79851f2 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:43.769165 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:43.769768 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:43.770051 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:43.770324 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] PATH_INFO: `/v3/projects/34dc6542be764622b304ee5f895041a2` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:43.771518 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] RBAC: Authorizing `identity:delete_project(project_id=34dc6542be764622b304ee5f895041a2)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:43.773459 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:43.784116 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=100 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:39:43.820760 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34dc6542be764622b304ee5f895041a2', 'request_id': 'req-2ca098f5-7f71-4a66-92e9-fd437d65776f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.828520 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34dc6542be764622b304ee5f895041a2', 'request_id': 'req-2ca098f5-7f71-4a66-92e9-fd437d65776f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.834146 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34dc6542be764622b304ee5f895041a2', 'request_id': 'req-2ca098f5-7f71-4a66-92e9-fd437d65776f'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.857607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] The token cache is being invalidate because project 34dc6542be764622b304ee5f895041a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:39:43.857607 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.858206 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.861019 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1449/2902] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:39:43 2022] DELETE /identity/v3/projects/34dc6542be764622b304ee5f895041a2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:39:43.865002 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:43.866702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:43.879683 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cf3b4-5723-4406-958e-810c83bfa8ea admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:43.920523 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:39:43.939223 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:43.939860 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:43.940139 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:43.940423 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] PATH_INFO: `/v3/users/cf751aab9be7453d85fe7f9a4bde2edd` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:43.941662 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf751aab9be7453d85fe7f9a4bde2edd)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:43.943600 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:43.988492 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf751aab9be7453d85fe7f9a4bde2edd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:43.995498 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf751aab9be7453d85fe7f9a4bde2edd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.003796 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf751aab9be7453d85fe7f9a4bde2edd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.004579 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf751aab9be7453d85fe7f9a4bde2edd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.017166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf751aab9be7453d85fe7f9a4bde2edd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.023383 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1454/2903] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:39:43 2022] DELETE /identity/v3/users/cf751aab9be7453d85fe7f9a4bde2edd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:39:44.203257 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:39:44.204759 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:39:44.219458 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca098f5-7f71-4a66-92e9-fd437d65776f admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:39:44.277604 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:39:44.278460 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:39:44.278755 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:39:44.279030 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] PATH_INFO: `/v3/projects/94fa04d4f22d45c5a7a7e080fb2645f9` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:39:44.280211 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94fa04d4f22d45c5a7a7e080fb2645f9)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:39:44.282390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:39:44.329179 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94fa04d4f22d45c5a7a7e080fb2645f9', 'request_id': 'req-a700fe5e-1890-48a0-a431-810161633dd1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.336577 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94fa04d4f22d45c5a7a7e080fb2645f9', 'request_id': 'req-a700fe5e-1890-48a0-a431-810161633dd1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.342583 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94fa04d4f22d45c5a7a7e080fb2645f9', 'request_id': 'req-a700fe5e-1890-48a0-a431-810161633dd1'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.353296 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] The token cache is being invalidate because project 94fa04d4f22d45c5a7a7e080fb2645f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:39:44.353614 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.354256 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:39:44.356298 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1450/2904] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:39:44 2022] DELETE /identity/v3/projects/94fa04d4f22d45c5a7a7e080fb2645f9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:39:54.019700 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:39:54.350385 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=76,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:41:31.006606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:41:31.008115 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:31.066382 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269a839-67d1-4f27-bffd-ac987c7880f7 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:41:31.114928 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:41:31.115565 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:41:31.115844 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:41:31.116118 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:41:31.117955 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:31.127702 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:41:31.129309 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:41:31.137915 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1455/2905] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:41:31 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:41:33.473472 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:41:33.474921 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:33.493695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700fe5e-1890-48a0-a431-810161633dd1 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:41:33.497672 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:41:33.498295 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:41:33.498608 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:41:33.498899 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:41:33.501400 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:33.585941 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:41:33.587425 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:41:33.600002 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1451/2906] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:41:33 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:41:34.791151 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:41:34.792657 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:34.804179 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc849219-6217-44b8-b020-6a343ab3b40a service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:41:34.807861 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:41:34.808463 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:41:34.808744 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:41:34.809015 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:41:34.810791 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:34.824606 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:41:34.826061 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:41:34.837431 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1456/2907] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:41:34 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:41:43.595029 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=34 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:41:44.832380 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=52 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:41:45.649699 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:41:45.658665 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3552e-6bb8-4a52-af80-11566de76091 service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:41:45.662354 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5455c117e349480ba8b8c020b767bec5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:41:45.662984 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] REQUEST_METHOD: `GET` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:41:45.663264 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:41:45.663534 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:41:45.665331 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:45.730669 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:41:45.733238 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:41:45.745345 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1452/2908] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:41:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:41:52.038495 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:41:52.039995 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:52.062724 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768e50e-3973-48da-a8de-d971cb8539ce service neutron] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:41:52.068351 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:41:52.069004 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:41:52.069284 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:41:52.069598 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] PATH_INFO: `/v3/users/43e543cfa3e1402e9694dcb095a26c00` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:41:52.070771 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e543cfa3e1402e9694dcb095a26c00)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:41:52.072705 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:41:52.125089 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e543cfa3e1402e9694dcb095a26c00'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.132487 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e543cfa3e1402e9694dcb095a26c00'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.139795 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e543cfa3e1402e9694dcb095a26c00'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.140403 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e543cfa3e1402e9694dcb095a26c00'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.153210 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e543cfa3e1402e9694dcb095a26c00'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.159176 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1457/2909] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:41:52 2022] DELETE /identity/v3/users/43e543cfa3e1402e9694dcb095a26c00 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:41:52.330765 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:41:52.341750 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d217c6-55e3-439f-8c09-0d0841e1846f service neutron] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:41:52.400329 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:41:52.400988 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:41:52.401274 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:41:52.401574 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] PATH_INFO: `/v3/projects/49576f2166cc4b7f9261bd7cbc8384f4` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:41:52.402772 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49576f2166cc4b7f9261bd7cbc8384f4)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:41:52.404695 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:41:52.451390 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49576f2166cc4b7f9261bd7cbc8384f4', 'request_id': 'req-dfd49552-e4c3-4f59-a32b-e97632deb484'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.459083 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49576f2166cc4b7f9261bd7cbc8384f4', 'request_id': 'req-dfd49552-e4c3-4f59-a32b-e97632deb484'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.464646 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49576f2166cc4b7f9261bd7cbc8384f4', 'request_id': 'req-dfd49552-e4c3-4f59-a32b-e97632deb484'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.476410 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] The token cache is being invalidate because project 49576f2166cc4b7f9261bd7cbc8384f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:41:52.476693 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.477326 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.478883 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1453/2910] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:41:52 2022] DELETE /identity/v3/projects/49576f2166cc4b7f9261bd7cbc8384f4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 22:41:52.482882 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:41:52.484281 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:52.497829 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ef8a6-104d-4794-97ef-c477af4d837a admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:41:52.554194 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:41:52.554765 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:41:52.555044 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:41:52.555314 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] PATH_INFO: `/v3/users/c8a0ea459aad47f18d7e03f546f123cd` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:41:52.556522 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a0ea459aad47f18d7e03f546f123cd)` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:41:52.558483 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:41:52.604413 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a0ea459aad47f18d7e03f546f123cd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.611275 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a0ea459aad47f18d7e03f546f123cd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.618544 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a0ea459aad47f18d7e03f546f123cd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.619156 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a0ea459aad47f18d7e03f546f123cd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.627774 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:41:52.633236 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.notifications [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a0ea459aad47f18d7e03f546f123cd'} {{(pid=91702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.640043 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1458/2911] 10.222.0.109 () {58 vars in 1229 bytes} [Thu Aug 4 22:41:52 2022] DELETE /identity/v3/users/c8a0ea459aad47f18d7e03f546f123cd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:41:52.810592 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] Authenticating user token {{(pid=91701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:41:52.812024 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.fernet_utils [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:41:52.825643 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd49552-e4c3-4f59-a32b-e97632deb484 admin admin] Validating token access rules against request {{(pid=91701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:41:52.847743 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:41:52.885314 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f59916fe4d6d4a6aac1fc1ff790ac163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee8a7cb4ed304a5598e10ca71a963013', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:41:52.885959 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:41:52.886234 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] SCRIPT_NAME: `/identity` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:41:52.886502 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] PATH_INFO: `/v3/projects/a54de9fb6e9e4d6880ae34c5b3779635` {{(pid=91701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:41:52.887644 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a54de9fb6e9e4d6880ae34c5b3779635)` {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:41:52.889576 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] RBAC: Authorization granted {{(pid=91701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:41:52.935278 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a54de9fb6e9e4d6880ae34c5b3779635', 'request_id': 'req-7b3de652-a3e5-41ec-92a4-fee1b62e4869'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.943932 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a54de9fb6e9e4d6880ae34c5b3779635', 'request_id': 'req-7b3de652-a3e5-41ec-92a4-fee1b62e4869'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.949688 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a54de9fb6e9e4d6880ae34c5b3779635', 'request_id': 'req-7b3de652-a3e5-41ec-92a4-fee1b62e4869'} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.960773 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] The token cache is being invalidate because project a54de9fb6e9e4d6880ae34c5b3779635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 22:41:52.961040 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.961681 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG keystone.notifications [None req-7b3de652-a3e5-41ec-92a4-fee1b62e4869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 22:41:52.963162 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: [pid: 91701|app: 0|req: 1454/2912] 10.222.0.109 () {58 vars in 1238 bytes} [Thu Aug 4 22:41:52 2022] DELETE /identity/v3/projects/a54de9fb6e9e4d6880ae34c5b3779635 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 22:42:02.636782 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:42:02.957844 dv-cinder-852211-1-iscsi devstack@keystone.service[91701]: DEBUG dbcounter [-] [91701] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 22:44:43.277763 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] Authenticating user token {{(pid=91702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 22:44:43.279166 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:44:43.347905 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8add9c0-ee11-456b-8624-a3aa52c17865 admin admin] Validating token access rules against request {{(pid=91702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 22:44:43.409032 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7e29-c869-4235-83a8-1a65582c2dff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0583af73d78a40deb2ca3af755928cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01df3a6495b04e38be92a133494b85cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 22:44:43.410249 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7e29-c869-4235-83a8-1a65582c2dff service placement] REQUEST_METHOD: `GET` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 22:44:43.410542 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7e29-c869-4235-83a8-1a65582c2dff service placement] SCRIPT_NAME: `/identity` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 22:44:43.410821 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7e29-c869-4235-83a8-1a65582c2dff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 22:44:43.412602 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.fernet_utils [None req-4cce7e29-c869-4235-83a8-1a65582c2dff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 22:44:43.423786 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce7e29-c869-4235-83a8-1a65582c2dff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 22:44:43.425389 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce7e29-c869-4235-83a8-1a65582c2dff service placement] RBAC: Authorization granted {{(pid=91702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 22:44:43.435672 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: [pid: 91702|app: 0|req: 1459/2913] 10.222.0.109 () {60 vars in 1370 bytes} [Thu Aug 4 22:44:43 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 22:44:53.431464 dv-cinder-852211-1-iscsi devstack@keystone.service[91702]: DEBUG dbcounter [-] [91702] Writing DB stats keystone:SELECT=58 {{(pid=91702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}